Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration: 5.626224935s FAIL (2025-08-11 22:00:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 5.626273748s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.........X...............X...........X.........] duration: 2.879698239s FAIL (2025-08-11 21:59:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.879726681s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.........X...............X...........X........] duration: 2.806268517s FAIL (2025-08-11 21:59:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.806296693s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration: 461.00383ms FAIL (2025-08-11 21:59:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 461.030869ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.........X...............X...........X.......] duration: 2.646352324s FAIL (2025-08-11 21:58:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.646385311s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 390.06141ms FAIL (2025-08-11 21:58:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 390.09656ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.........X...............X...........X......] duration: 2.721406961s FAIL (2025-08-11 21:58:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.721477249s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X...............X.........X...............X...........X.....] duration: 2.762323815s FAIL (2025-08-11 21:57:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.762356015s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 429.940778ms FAIL (2025-08-11 21:57:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 429.972473ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X...............X.........X...............X...........X....] duration: 2.563539937s FAIL (2025-08-11 21:57:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.56366483s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 391.355565ms FAIL (2025-08-11 21:57:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 391.394831ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 456.006849ms FAIL (2025-08-11 21:56:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 456.058195ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X...............X.........X...............X...........X...] duration: 2.944214949s FAIL (2025-08-11 21:56:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.944254138s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X...............X.........X...............X...........X..] duration: 2.777766369s FAIL (2025-08-11 21:55:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 2.777797744s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 404.784085ms FAIL (2025-08-11 21:55:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 404.814281ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X...............X.........X...............X...........X.] duration: 3.508574664s FAIL (2025-08-11 21:55:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 3.508666524s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration: 396.673711ms FAIL (2025-08-11 21:54:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 396.705271ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [......................................XXX...................] duration: 433.46144ms FAIL (2025-08-11 21:54:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 433.492696ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X...............X.........X...............X...........X] duration: 1.88109249s FAIL (2025-08-11 21:54:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 11 21:54:01.234852081 (76h28m23.193949341s) [upload=79721]: Starting upload Aug 11 21:54:01.234930221 (76h28m23.194027487s) [upload=79721]: Done starting upload Aug 11 21:54:01.235253478 (76h28m23.194350726s) [upload=79721]: Uploading segments... Aug 11 21:54:01.235375450 (76h28m23.194472705s) [upload=79721]: Got next segment. Inline: false Aug 11 21:54:01.235444098 (76h28m23.19454135s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:54:01.235451185 (76h28m23.194548425s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:54:01.441395491 (76h28m23.400492771s) [seg_pos={0 0}, upload=79721]: Begin upload segment... Aug 11 21:54:01.441467088 (76h28m23.400564337s) [seg_pos={0 0}, upload=79721]: Done begin upload segment. Aug 11 21:54:01.441505082 (76h28m23.400602336s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441521704 (76h28m23.40061896s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.441639547 (76h28m23.4007368s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442132798 (76h28m23.401230057s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442175796 (76h28m23.401273049s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442555959 (76h28m23.401653215s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.442664733 (76h28m23.401761988s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443004162 (76h28m23.402101419s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443177996 (76h28m23.402275255s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443415663 (76h28m23.402512914s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.443807796 (76h28m23.402905057s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444015865 (76h28m23.403113119s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444409995 (76h28m23.403507257s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444441587 (76h28m23.403538843s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444885002 (76h28m23.403982261s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.444905020 (76h28m23.404002279s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445334964 (76h28m23.404432197s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:01.445408898 (76h28m23.404506153s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Uploading piece... Aug 11 21:54:02.034967579 (76h28m23.994064838s) [upload=79721]: Next returned nil segment Aug 11 21:54:02.034992943 (76h28m23.994090202s) [upload=79721]: Waiting for error group managing segments... Aug 11 21:54:02.209416862 (76h28m24.168514121s) [noise=true , address=104.36.82.37:4681 , piece_id=Z5XENGR6GILWN4UZNKUZS6MPHEFN7L4M5UL24OL45KZHXCXSDGFQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.210260243 (76h28m24.169357504s) [noise=true , address=104.36.82.58:4272 , piece_id=LDITA3UHTWBPJ6UDGNNQFYI7BIFKF7Q6CN223477HRHVO3DL3C5Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.231522717 (76h28m24.190619973s) [noise=true , address=104.36.82.19:4415 , piece_id=GBC3ZHZZDYKNCGNCO3FTZ4G5IQQDTE33K3O52WTIOLZHIXMUOBRQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.235173584 (76h28m24.194270834s) [noise=true , address=104.36.82.23:4463 , piece_id=AXTJC2J2RTXA23ZLJPIXI3GSKMCN4NZJ2BPLEELY6377POVBMBHA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.254557988 (76h28m24.213655238s) [noise=true , address=79.127.219.38:4001 , piece_id=LFHQ5HAJECJBZOQOUZEA2OUHWEKYYNXWLZ6QGOTU5KPFKDO4YWUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.262318927 (76h28m24.221416168s) [noise=true , address=104.36.82.15:4263 , piece_id=LLLNBFVA5QLB3WAPBCRTMN5OJP2XVVFTNIHEF3DG4O2FUZTNHDJA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.263899826 (76h28m24.222997075s) [noise=true , address=104.36.82.21:4498 , piece_id=JGYX4CUGW5QHMU5IS3N337ZP7QSGCKOZBYN5JQYQ22ZHSJ3D7AUQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.275982498 (76h28m24.235079739s) [noise=true , address=104.36.82.15:4277 , piece_id=LAYK37RHKEOQWU2QCAUICTLI2EJTFGCX7KXL4E7LWAMT7XUF4NFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.281527540 (76h28m24.240624791s) [noise=true , address=104.36.82.35:4583 , piece_id=OQ2ST6QR3NQ7GMCQKRGM6V6NAFXUXH3K7K7VURMRVBPMMDOPEG4Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.283957954 (76h28m24.243055212s) [noise=true , address=104.36.82.16:4305 , piece_id=PHQWGMQN7HCCOYI5XRLIIIZ2KUKQ45H6XSJEVL5LPFXAX4NPNWVA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.292448867 (76h28m24.251546124s) [noise=true , address=104.36.82.9:4019 , piece_id=EMTTW6KYPKCYVPPL7AUTNZOSXRPK7S362OAWRARMXUREWNTK6VEQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.299004582 (76h28m24.258101837s) [noise=true , address=79.127.219.48:4086 , piece_id=JMHKHXCCNPOITCSDBDFNFOCONO4VKHB7BUXTT3LIVHE4362FYVYA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.342075148 (76h28m24.301172403s) [noise=true , address=104.36.82.16:4307 , piece_id=URU4SGWQSEMDTYTFOOVWX2NPZTDF6VZNAPRBP3JIQEL7L46SOBFA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402369958 (76h28m24.361467199s) [noise=true , address=104.36.82.11:4137 , piece_id=U4JFH2WOGZ3DPYCZVLPR2M4D242VJ5NI33KJ6TFSNKDSBQOZU5ZA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: <nil> Aug 11 21:54:02.402435765 (76h28m24.361533013s) [seg_pos={0 0}, upload=79721]: Segment reached optimal threshold of 14 pieces. Aug 11 21:54:02.402939390 (76h28m24.362036646s) [noise=true , address=64.49.13.138:4484 , piece_id=HGESUIVWP6DRK7KRLPAN74FJBBSWNS4F7KMTALLYHUP5LZ4EHTTA, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eJA68UbWuVhgmDLo4kHe3awpGfziwMRrbajaBEGNLua5VTZNQ): protocol: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40614->64.49.13.138:4484: use of closed network connection Aug 11 21:54:02.403119103 (76h28m24.36221636s) [noise=true , address=156.146.43.225:4441 , piece_id=FVZE3CYMBSJ5C6AVYTTMS3O4TA73WHPWTK3KWEIAUWNQP5IVW3JQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12D2fbu6KYkVaLhiGXS8RBnExWrprgf5j6eaSHu4iYbdm8rAKfY): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403378236 (76h28m24.362475485s) [noise=true , address=79.127.201.215:4344 , piece_id=732VK4X77325F7ARFUFPMF2YC3NB6A4DJIQ3YWIE53KRNVQXKYQQ, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RXve6Cd4pYRqUySbbaUKk9Xz3qjMPDAit9XRveV5DL5p3iVwU): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403480004 (76h28m24.362577255s) [noise=true , address=104.36.82.2:4760 , piece_id=D6Q4U4C3Z2L3YIVAABHKVCCPOOIZQJO2BQBXQ3SHEXEQDHVUSC2Q, seg_pos={0 0}, upload=79721]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pgY3KuDopCrLXxvbVeNowHD9qSh2Gz7qcgUR1GpsxFkuoHhzB): protocol: expected piece hash; context canceled; EOF Aug 11 21:54:02.403496453 (76h28m24.362593709s) [seg_pos={0 0}, upload=79721]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:54:02.403664331 (76h28m24.362761581s) [upload=79721]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:54:02.403671940 (76h28m24.362769197s) [upload=79721]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:54:02.575608710 (76h28m24.534705966s) [upload=79721]: Done uploading segments... Aug 11 21:54:02.575636492 (76h28m24.534733732s) [upload=79721]: Upload finished. err: <nil> Duration: 1.881199052s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration: 410.867225ms FAIL (2025-08-11 21:53:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 410.899817ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X...............X.........X...............X...........] duration: 2.665071482s FAIL (2025-08-11 21:53:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.665111058s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [........................................XXX.................] duration: 393.125657ms FAIL (2025-08-11 21:53:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 393.164615ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X...............X.........X...............X..........] duration: 2.493175169s FAIL (2025-08-11 21:53:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.493206924s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 455.401743ms FAIL (2025-08-11 21:52:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 455.435837ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X...............X.........X...............X.........] duration: 3.024110984s FAIL (2025-08-11 21:52:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 3.024165581s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 439.541014ms FAIL (2025-08-11 21:52:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 439.57802ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X...............X.........X...............X........] duration: 2.84044639s FAIL (2025-08-11 21:51:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.840471149s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 536.869706ms FAIL (2025-08-11 21:51:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 536.908097ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.........X...............X.........X...............X.......] duration: 2.73196886s FAIL (2025-08-11 21:51:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.732050888s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.........X...............X.........X...............X......] duration: 3.452970838s FAIL (2025-08-11 21:50:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 3.453013186s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 466.285005ms FAIL (2025-08-11 21:49:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 466.419303ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.........X...............X.........X...............X.....] duration: 3.162383582s FAIL (2025-08-11 21:49:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 3.162413558s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.........X...............X.........X...............X....] duration: 2.628649427s FAIL (2025-08-11 21:49:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.628677679s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 393.312855ms FAIL (2025-08-11 21:48:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 393.345793ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X.........X...............X.........X...............X...] duration: 2.802346527s FAIL (2025-08-11 21:48:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.802380991s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 490.911605ms FAIL (2025-08-11 21:48:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 490.946374ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X.........X...............X.........X...............X..] duration: 2.4511814s FAIL (2025-08-11 21:48:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 2.451222285s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 367.522086ms FAIL (2025-08-11 21:47:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 367.553164ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X.........X...............X.........X...............X.] duration: 3.282188833s FAIL (2025-08-11 21:47:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 3.282246347s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 386.250793ms FAIL (2025-08-11 21:47:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 386.286094ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X.........X...............X.........X...............X] duration: 367.169244ms FAIL (2025-08-11 21:46:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 21:46:48.105628159 (76h21m10.064725415s) [upload=79595]: Starting upload Aug 11 21:46:48.105701877 (76h21m10.064799123s) [upload=79595]: Done starting upload Aug 11 21:46:48.106061459 (76h21m10.0651587s) [upload=79595]: Uploading segments... Aug 11 21:46:48.106147967 (76h21m10.065245221s) [upload=79595]: Got next segment. Inline: false Aug 11 21:46:48.106205234 (76h21m10.065302697s) [upload=79595]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:46:48.106214217 (76h21m10.065311465s) [upload=79595]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:46:48.195345971 (76h21m10.154443219s) [upload=79595]: Next segment err: metaclient: Too Many Requests Aug 11 21:46:48.195369798 (76h21m10.154467043s) [upload=79595]: Done uploading segments... Aug 11 21:46:48.195404017 (76h21m10.154501256s) [upload=79595]: Upload finished. err: metaclient: Too Many Requests Duration: 367.194299ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 530.018008ms FAIL (2025-08-11 21:46:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 530.047271ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 408.045271ms FAIL (2025-08-11 21:46:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 408.078175ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X.........X...............X.........X...............] duration: 3.621028616s FAIL (2025-08-11 21:45:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9a43a61b148a4c46ea4da5d61c37d408ca6ffa3422b5de75670051160da20bbf != before dcc7130f5e77e281c9f0c667d9bb7a7a2ff7a6b2c5a87fa039d2563122b3eeba ======LOGS====== Aug 11 21:36:22.337717042 (76h10m44.296814286s) [upload=79403]: Starting upload Aug 11 21:36:22.337798041 (76h10m44.296895286s) [upload=79403]: Done starting upload Aug 11 21:36:22.337924680 (76h10m44.297022031s) [upload=79403]: Uploading segments... Aug 11 21:36:22.338121095 (76h10m44.297218341s) [upload=79403]: Got next segment. Inline: false Aug 11 21:36:22.338168728 (76h10m44.297265973s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:36:22.338173731 (76h10m44.297270976s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:36:22.537646727 (76h10m44.49674398s) [seg_pos={0 0}, upload=79403]: Begin upload segment... Aug 11 21:36:22.537698911 (76h10m44.496796149s) [seg_pos={0 0}, upload=79403]: Done begin upload segment. Aug 11 21:36:22.537725223 (76h10m44.496822458s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.537973682 (76h10m44.497070926s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538352402 (76h10m44.49744965s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538680364 (76h10m44.49777761s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538986470 (76h10m44.498083715s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539303691 (76h10m44.498400948s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539618714 (76h10m44.49871596s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540065302 (76h10m44.499162547s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540412988 (76h10m44.499510244s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540823800 (76h10m44.49992106s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543124073 (76h10m44.502221328s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543516192 (76h10m44.502613453s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543921681 (76h10m44.503018929s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544281059 (76h10m44.503378309s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544599869 (76h10m44.503697125s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544948104 (76h10m44.504045355s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545286620 (76h10m44.504383868s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545593034 (76h10m44.504690282s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:23.107074136 (76h10m45.066171375s) [upload=79403]: Next returned nil segment Aug 11 21:36:23.107096985 (76h10m45.06619423s) [upload=79403]: Waiting for error group managing segments... Aug 11 21:36:23.281673303 (76h10m45.240770549s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.282484874 (76h10m45.241582108s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.295798412 (76h10m45.254895651s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.296653206 (76h10m45.255750457s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.314593660 (76h10m45.273690899s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.321810248 (76h10m45.280907486s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.322649530 (76h10m45.281746776s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.325045899 (76h10m45.284143139s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.331049467 (76h10m45.290146715s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.338037895 (76h10m45.297135132s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.343757912 (76h10m45.302855153s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.344456504 (76h10m45.303553743s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.364712044 (76h10m45.32380929s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391020329 (76h10m45.350117561s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391054738 (76h10m45.350151982s) [seg_pos={0 0}, upload=79403]: Segment reached optimal threshold of 14 pieces. Aug 11 21:36:23.391272958 (76h10m45.350370196s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr): protocol: expected piece hash; context canceled; EOF Aug 11 21:36:23.391309551 (76h10m45.350406783s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ctmSF1aMQasuXByFW8DppDantREmmWL2WUjmKGW5ZULVePyvm): protocol: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection Aug 11 21:36:23.391364354 (76h10m45.350461585s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126wi7mavkxXLAWpoK1VP6QX2QPddsydzkMNRJc5uDU3LrCRVY2): protocol: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection Aug 11 21:36:23.391463684 (76h10m45.350560924s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection Aug 11 21:36:23.391478341 (76h10m45.35057558s) [seg_pos={0 0}, upload=79403]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:36:23.391564994 (76h10m45.350662254s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:36:23.391572564 (76h10m45.350669824s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:36:24.219148218 (76h10m46.178245474s) [upload=79403]: Done uploading segments... Aug 11 21:36:24.219191596 (76h10m46.178288851s) [upload=79403]: Upload finished. err: <nil> Duration: 3.621061197s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 432.611366ms FAIL (2025-08-11 21:45:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 432.647813ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 443.201027ms FAIL (2025-08-11 21:45:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 443.235184ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X.........X...............X.........X..............] duration: 3.074774909s FAIL (2025-08-11 21:45:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9a43a61b148a4c46ea4da5d61c37d408ca6ffa3422b5de75670051160da20bbf != before dcc7130f5e77e281c9f0c667d9bb7a7a2ff7a6b2c5a87fa039d2563122b3eeba ======LOGS====== Aug 11 21:36:22.337717042 (76h10m44.296814286s) [upload=79403]: Starting upload Aug 11 21:36:22.337798041 (76h10m44.296895286s) [upload=79403]: Done starting upload Aug 11 21:36:22.337924680 (76h10m44.297022031s) [upload=79403]: Uploading segments... Aug 11 21:36:22.338121095 (76h10m44.297218341s) [upload=79403]: Got next segment. Inline: false Aug 11 21:36:22.338168728 (76h10m44.297265973s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:36:22.338173731 (76h10m44.297270976s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:36:22.537646727 (76h10m44.49674398s) [seg_pos={0 0}, upload=79403]: Begin upload segment... Aug 11 21:36:22.537698911 (76h10m44.496796149s) [seg_pos={0 0}, upload=79403]: Done begin upload segment. Aug 11 21:36:22.537725223 (76h10m44.496822458s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.537973682 (76h10m44.497070926s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538352402 (76h10m44.49744965s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538680364 (76h10m44.49777761s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538986470 (76h10m44.498083715s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539303691 (76h10m44.498400948s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539618714 (76h10m44.49871596s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540065302 (76h10m44.499162547s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540412988 (76h10m44.499510244s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540823800 (76h10m44.49992106s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543124073 (76h10m44.502221328s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543516192 (76h10m44.502613453s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543921681 (76h10m44.503018929s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544281059 (76h10m44.503378309s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544599869 (76h10m44.503697125s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544948104 (76h10m44.504045355s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545286620 (76h10m44.504383868s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545593034 (76h10m44.504690282s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:23.107074136 (76h10m45.066171375s) [upload=79403]: Next returned nil segment Aug 11 21:36:23.107096985 (76h10m45.06619423s) [upload=79403]: Waiting for error group managing segments... Aug 11 21:36:23.281673303 (76h10m45.240770549s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.282484874 (76h10m45.241582108s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.295798412 (76h10m45.254895651s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.296653206 (76h10m45.255750457s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.314593660 (76h10m45.273690899s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.321810248 (76h10m45.280907486s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.322649530 (76h10m45.281746776s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.325045899 (76h10m45.284143139s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.331049467 (76h10m45.290146715s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.338037895 (76h10m45.297135132s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.343757912 (76h10m45.302855153s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.344456504 (76h10m45.303553743s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.364712044 (76h10m45.32380929s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391020329 (76h10m45.350117561s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391054738 (76h10m45.350151982s) [seg_pos={0 0}, upload=79403]: Segment reached optimal threshold of 14 pieces. Aug 11 21:36:23.391272958 (76h10m45.350370196s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr): protocol: expected piece hash; context canceled; EOF Aug 11 21:36:23.391309551 (76h10m45.350406783s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ctmSF1aMQasuXByFW8DppDantREmmWL2WUjmKGW5ZULVePyvm): protocol: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection Aug 11 21:36:23.391364354 (76h10m45.350461585s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126wi7mavkxXLAWpoK1VP6QX2QPddsydzkMNRJc5uDU3LrCRVY2): protocol: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection Aug 11 21:36:23.391463684 (76h10m45.350560924s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection Aug 11 21:36:23.391478341 (76h10m45.35057558s) [seg_pos={0 0}, upload=79403]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:36:23.391564994 (76h10m45.350662254s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:36:23.391572564 (76h10m45.350669824s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:36:24.219148218 (76h10m46.178245474s) [upload=79403]: Done uploading segments... Aug 11 21:36:24.219191596 (76h10m46.178288851s) [upload=79403]: Upload finished. err: <nil> Duration: 3.07485411s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 408.734101ms FAIL (2025-08-11 21:44:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 408.769479ms core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.........X.........X...............X.........X.............] duration: 2.896189186s FAIL (2025-08-11 21:44:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 9a43a61b148a4c46ea4da5d61c37d408ca6ffa3422b5de75670051160da20bbf != before dcc7130f5e77e281c9f0c667d9bb7a7a2ff7a6b2c5a87fa039d2563122b3eeba ======LOGS====== Aug 11 21:36:22.337717042 (76h10m44.296814286s) [upload=79403]: Starting upload Aug 11 21:36:22.337798041 (76h10m44.296895286s) [upload=79403]: Done starting upload Aug 11 21:36:22.337924680 (76h10m44.297022031s) [upload=79403]: Uploading segments... Aug 11 21:36:22.338121095 (76h10m44.297218341s) [upload=79403]: Got next segment. Inline: false Aug 11 21:36:22.338168728 (76h10m44.297265973s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 21:36:22.338173731 (76h10m44.297270976s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 21:36:22.537646727 (76h10m44.49674398s) [seg_pos={0 0}, upload=79403]: Begin upload segment... Aug 11 21:36:22.537698911 (76h10m44.496796149s) [seg_pos={0 0}, upload=79403]: Done begin upload segment. Aug 11 21:36:22.537725223 (76h10m44.496822458s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.537973682 (76h10m44.497070926s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538352402 (76h10m44.49744965s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538680364 (76h10m44.49777761s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.538986470 (76h10m44.498083715s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539303691 (76h10m44.498400948s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.539618714 (76h10m44.49871596s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540065302 (76h10m44.499162547s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540412988 (76h10m44.499510244s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.540823800 (76h10m44.49992106s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543124073 (76h10m44.502221328s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543516192 (76h10m44.502613453s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.543921681 (76h10m44.503018929s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544281059 (76h10m44.503378309s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544599869 (76h10m44.503697125s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.544948104 (76h10m44.504045355s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545286620 (76h10m44.504383868s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:22.545593034 (76h10m44.504690282s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Uploading piece... Aug 11 21:36:23.107074136 (76h10m45.066171375s) [upload=79403]: Next returned nil segment Aug 11 21:36:23.107096985 (76h10m45.06619423s) [upload=79403]: Waiting for error group managing segments... Aug 11 21:36:23.281673303 (76h10m45.240770549s) [noise=true , address=104.36.82.34:4579 , piece_id=466MNF3GFPSO2CJYP6HPNDQSNA24NTWK7Q6SKRA5COQGPIA4YCAA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.282484874 (76h10m45.241582108s) [noise=true , address=104.36.82.44:4899 , piece_id=W4SJXZ2UZ4IRDZAGIUDUEUPRCSQWWNSD53W4GWIYDNDKHJRMKBKA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.295798412 (76h10m45.254895651s) [noise=true , address=77.90.31.12:4594 , piece_id=F6YNJUSM4ZYYX2WJAYDIX6EH4R6NJB3KV2UOPLQY5UANKP3LXX6A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.296653206 (76h10m45.255750457s) [noise=true , address=104.36.82.39:4737 , piece_id=3NYTDRXRJXJK7SNT6PNUODNGDR7AAA4ZF32P4YMNLR5HXVFDD6BQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.314593660 (76h10m45.273690899s) [noise=true , address=104.36.82.37:4654 , piece_id=BZ7HJYHZUPQJH467KALD5YSZSNFLBS57TRJ4KWCRATT7WQYKADEQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.321810248 (76h10m45.280907486s) [noise=true , address=104.36.82.38:4690 , piece_id=VPKA3GBE7AYAIYXYZ3L6YQWCGU3MY2ENXS5MXOE4TALXLDRWXUOQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.322649530 (76h10m45.281746776s) [noise=true , address=104.36.82.44:4882 , piece_id=CFBIJCUIKRM2MGCS536LXPBCXJULQCIIOEA5USMMYAXNHPC73CSQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.325045899 (76h10m45.284143139s) [noise=true , address=104.36.82.6:4891 , piece_id=QLEU2OGUO5JQPZJNDMCPBAGERKSSSXX2337T4VREFXF66JIOK4GQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.331049467 (76h10m45.290146715s) [noise=true , address=104.36.82.62:4439 , piece_id=C7XUK2GECEDN6DKV4D2T2IC3PNUG3DLZTGXETVCNQXLUK2U2T3MQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.338037895 (76h10m45.297135132s) [noise=true , address=104.36.82.59:4326 , piece_id=43SINEDHR326FWPOJFJILRCXYHIBECIAFG4BA6QDNXRPOGUPFKUQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.343757912 (76h10m45.302855153s) [noise=true , address=104.36.82.36:4625 , piece_id=LB53WEZ4OH7ZJBNLL23XKTISQ3FD3UZMJZNKTWKIYTXBNIRAPKCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.344456504 (76h10m45.303553743s) [noise=true , address=104.36.82.35:4610 , piece_id=62Y2SORCGHR4AG5IL34FVWCOBNEHAPPLMYHMYIAE7DL7F6EAJL2A, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.364712044 (76h10m45.32380929s) [noise=true , address=77.90.31.13:4653 , piece_id=Q5BOVX4OTHGUAXDAJS4UJJ52R23R5BIDAJP3BVZERU2JZZYFBCEA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391020329 (76h10m45.350117561s) [noise=true , address=104.36.82.58:4298 , piece_id=3Y5YEXO4ZRDZAXMPKWHMBTWXMI6NOF44EQWSBSPFAKFU3IIPMQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: <nil> Aug 11 21:36:23.391054738 (76h10m45.350151982s) [seg_pos={0 0}, upload=79403]: Segment reached optimal threshold of 14 pieces. Aug 11 21:36:23.391272958 (76h10m45.350370196s) [noise=true , address=104.36.82.19:4429 , piece_id=ENOZWK2U27PW65XQC5TKRVKA3JRNNQFVIEI73EWVOYXN37GPZCKQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr): protocol: expected piece hash; context canceled; EOF Aug 11 21:36:23.391309551 (76h10m45.350406783s) [noise=true , address=104.36.82.46:4503 , piece_id=KVFA2XS7CHKVDMRAFTTYBHQHV7UDKQZ3BEDHKXDBBQQ7HDL5SBCA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ctmSF1aMQasuXByFW8DppDantREmmWL2WUjmKGW5ZULVePyvm): protocol: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60594->104.36.82.46:4503: use of closed network connection Aug 11 21:36:23.391364354 (76h10m45.350461585s) [noise=true , address=64.49.13.139:4474 , piece_id=DB7SO43YPY6E5BRWUZVNBLGQY4L5H2IPNPWEN2VTBMKTZ7BISRNA, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126wi7mavkxXLAWpoK1VP6QX2QPddsydzkMNRJc5uDU3LrCRVY2): protocol: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46658->64.49.13.139:4474: use of closed network connection Aug 11 21:36:23.391463684 (76h10m45.350560924s) [noise=true , address=64.49.13.138:4488 , piece_id=3VONVI2GN3TQD3EYYWPIPHRPFFGWLIO45ZZ6MCIJXBDE5G6EFQVQ, seg_pos={0 0}, upload=79403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36392->64.49.13.138:4488: use of closed network connection Aug 11 21:36:23.391478341 (76h10m45.35057558s) [seg_pos={0 0}, upload=79403]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 21:36:23.391564994 (76h10m45.350662254s) [upload=79403]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 21:36:23.391572564 (76h10m45.350669824s) [upload=79403]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 21:36:24.219148218 (76h10m46.178245474s) [upload=79403]: Done uploading segments... Aug 11 21:36:24.219191596 (76h10m46.178288851s) [upload=79403]: Upload finished. err: <nil> Duration: 2.89622135s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 625.114487ms FAIL (2025-08-11 21:44:05 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 20) ======LOGS====== , upload=79498]: Uploading piece... Aug 11 21:41:28.492965827 (76h15m50.452063074s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.493304523 (76h15m50.45240177s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.494260922 (76h15m50.453358182s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495332470 (76h15m50.45442973s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.495970910 (76h15m50.45506818s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.496444509 (76h15m50.455541767s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Uploading piece... Aug 11 21:41:28.620030725 (76h15m50.579127958s) [upload=79498]: Next returned nil segment Aug 11 21:41:28.620048735 (76h15m50.579145981s) [upload=79498]: Waiting for error group managing segments... Aug 11 21:41:28.629298196 (76h15m50.588395448s) [noise=true , address=89.149.201.147:4016 , piece_id=27TYWWB7DUSTCEBKY46KCGXZUDBQMVTZIRJF37ZEYX4SEHZEUB2A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630214646 (76h15m50.589311906s) [noise=true , address=81.171.14.70:4039 , piece_id=I4GSRTDSX7CBBNDB63PMXGKIXS5IDPFTVMMJP7BL5ML6J3O52XCQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.630974374 (76h15m50.590071607s) [noise=true , address=89.149.201.147:4005 , piece_id=POJTEBZVVTNOMYD62QKE5CA2JPMM2BTDFMXAE6MK7KDLPFLWJKDQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.631729297 (76h15m50.59082653s) [noise=true , address=81.171.14.70:4031 , piece_id=B5RDSPMGIN6EDC7JZ5MOQ6F7XTN4BDD63PJ6Y4BPCKNLA6ZONLJQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.632392232 (76h15m50.591489483s) [noise=true , address=89.149.201.147:4003 , piece_id=LTAMY6LJ34WAXJ5IZAQ2VAXEAZGYHCRWKKRXP4PN4PKEPQBFMZ6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633009190 (76h15m50.592106448s) [noise=true , address=89.149.201.147:4009 , piece_id=CGCMKBQH7CQUJKPBGGWJB33LZVZYQJPIUIRJBO44P2EVSLFZPQTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.633532114 (76h15m50.592629361s) [noise=true , address=89.149.201.147:4021 , piece_id=3LUHG7ADYE2OQRBET3GCP2DEKRNUT2RDYRNQV3FAQWRF6U3NRNVA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634209732 (76h15m50.593306975s) [noise=true , address=89.149.201.147:4004 , piece_id=YKYH4GLKA3TO7E6AZPDTYTWL7K7A6THQGPEDY2UXIH5MMOIAUUQQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634562268 (76h15m50.593659511s) [noise=true , address=89.149.201.147:4024 , piece_id=DMQWOG3ZOAIATO37THYSHC3YY5EODTQOBSOBOWNTOAEOJSYBDC5Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.634797625 (76h15m50.593894863s) [noise=true , address=81.171.14.70:4036 , piece_id=MUE2QGEW4QHHYZZQPDJ253IWRVVZNKDICGTO657R72DD2CRS3DTA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635103025 (76h15m50.594200265s) [noise=true , address=89.149.201.147:4020 , piece_id=OZ4U5DM5GPSGBKHADXRNNWPCVB37SCGAXS4VR2OQ4ZS7CXLVDI2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.635505276 (76h15m50.594602525s) [noise=true , address=89.149.201.147:4007 , piece_id=KWNJ5GF7SVCHEBMMNSYKG5UJCKGA4UTWV4OPGG6QBU4ALHY5FVFQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637184868 (76h15m50.596282123s) [noise=true , address=81.171.14.70:4038 , piece_id=WIO3LVAALYRXXYWQ4BIXOJCWK75VBB2BPBOVOEJVFT5BMMVSZN7Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.637789402 (76h15m50.596886659s) [noise=true , address=81.171.14.70:4047 , piece_id=5T62LWBAD3P6IPNENO5AQOT7YWJDA3MOFIKGKCX5FSU7FVBIF7ZA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.638445296 (76h15m50.59754255s) [noise=true , address=81.171.14.70:4042 , piece_id=657AR4BS4S4AB2A6V7IE2ERHNPSMIHEIY4IJPDQNX2YIBKHIWAIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.639922774 (76h15m50.59902003s) [noise=true , address=89.149.201.147:4012 , piece_id=FDMOUVSQRDLPEC2H7B2IGH7QQNBCERWINCOXYBC77SIOW5WRMQOQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.644334896 (76h15m50.603432139s) [noise=true , address=89.149.201.147:4018 , piece_id=ANO54TXCLWSKCTKB52PH4KFLLCOSXI6VJMPPO64OED7IJKXXMNCA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.646142616 (76h15m50.605239859s) [noise=true , address=89.149.201.147:4023 , piece_id=NTHLH3WC36UIIA2UWY7IHVBIMBAQKHCFI7HNFBQ4LO2TQ5OCDQBQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.649535498 (76h15m50.608632744s) [noise=true , address=89.149.201.147:4014 , piece_id=6EF7PHIL7PH4XFRTZ2HHUSP4JHT3WTW2KKW45BKDZLRVC6MXSTTQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:41:28.655767760 (76h15m50.614864995s) [noise=true , address=81.171.14.70:4029 , piece_id=J5DMN5H5JJ47YWXUJL6E6YD4MSRCX3K65CNZLSUJIN4H37PCYAFA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: <nil> Aug 11 21:42:28.434334827 (76h16m50.393432061s) [noise=true , address=185.167.204.206:4069 , piece_id=FY2X22DHFMNEEROLEHPJSKGIEHNO7SDKABXC3X5KMNIOWWKOLD6A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 139.59.144.22:53548->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434426322 (76h16m50.393523561s) [noise=true , address=185.167.204.205:4016 , piece_id=X3YYYHEBGMKNRDIXBDJIKFVNJTL5AZ7RCDBJ3OIRRTGY5J4KR47A, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ, address:185.167.204.205:4016): protocol: rpc: dial tcp 185.167.204.205:4016: i/o timeout; context deadline exceeded Aug 11 21:42:28.434525151 (76h16m50.393622401s) [noise=true , address=185.167.204.206:4038 , piece_id=M733YR3IEHWWKXMSPAFW7S5IHACALTA34TUA4HOARAPB4SV6DB2Q, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 139.59.144.22:59314->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434585233 (76h16m50.393682483s) [noise=true , address=185.167.204.206:4072 , piece_id=7TYCEZEVBNDDHBQCANKP3EFVJDTMKL3AS6LFEQMFJP2OST4J3SOA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v, address:185.167.204.206:4072): protocol: write tcp 139.59.144.22:57258->185.167.204.206:4072: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434738889 (76h16m50.393836141s) [noise=true , address=185.167.204.206:4057 , piece_id=EJTXN3KLFTQFZKTKI7MWSIWB6SJUGLG3NIQNZJFRDJ7GAIU32PIA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT, address:185.167.204.206:4057): protocol: rpc: dial tcp 185.167.204.206:4057: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434784220 (76h16m50.393881471s) [noise=true , address=185.167.204.205:4024 , piece_id=QEL34FCC7WNOFKELG6TYRQS7BGQ4JDNCYIHXPVQQK7R7JIMTUSXA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: write tcp 139.59.144.22:38718->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434806832 (76h16m50.393904078s) [noise=true , address=185.167.204.205:4012 , piece_id=HH6IMUQRFO6A4CRYYUTBMA6JSJ5DNRDXYAMLZ7SIOPMZRWXB56LQ, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 139.59.144.22:37662->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434839249 (76h16m50.393936494s) [noise=true , address=185.167.204.206:4061 , piece_id=JDGGWXLAWYTCACJQKPY6WH64K7NAOQJZ6VUP2ZNBKPFJNPHPD6XA, seg_pos={0 0}, upload=79498]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 139.59.144.22:56270->185.167.204.206:4061: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 11 21:42:28.434886690 (76h16m50.393983946s) [seg_pos={0 0}, upload=79498]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil> Aug 11 21:42:28.464204638 (76h16m50.423301879s) [upload=79498]: Done uploading segments... Aug 11 21:42:28.464242167 (76h16m50.423339426s) [upload=79498]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20) Duration: 625.151358ms core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)