Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.614494866s FAIL (2025-09-07 01:27:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.614524357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.498615792s FAIL (2025-09-07 01:27:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.498656195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.548593821s FAIL (2025-09-07 01:26:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.548624655s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.469014387s FAIL (2025-09-07 01:25:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.469108634s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.291320729s FAIL (2025-09-07 01:24:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.291421011s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.559766623s FAIL (2025-09-07 01:24:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.559802603s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.627016717s FAIL (2025-09-07 01:23:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.62705259s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.32208844s FAIL (2025-09-07 01:22:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.322212809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.354854531s FAIL (2025-09-07 01:22:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.354891559s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.559689769s FAIL (2025-09-07 01:21:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.559723985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.618540671s FAIL (2025-09-07 01:20:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.618629443s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.576343757s FAIL (2025-09-07 01:20:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576413769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.275003628s FAIL (2025-09-07 01:19:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.275041823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.402037826s FAIL (2025-09-07 01:18:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.402154452s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.438291298s FAIL (2025-09-07 01:17:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.438324943s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.317199786s FAIL (2025-09-07 01:17:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.317238684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.384923517s FAIL (2025-09-07 01:16:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.384961606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.471899611s FAIL (2025-09-07 01:15:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.471936648s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.389391601s FAIL (2025-09-07 01:13:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.389425269s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.381636485s FAIL (2025-09-07 01:12:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.38166876s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.484004533s FAIL (2025-09-07 01:11:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.484037688s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.385203188s FAIL (2025-09-07 01:10:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.385238637s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.34337949s FAIL (2025-09-07 01:09:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.343409286s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.394594466s FAIL (2025-09-07 01:08:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.394644156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.467240355s FAIL (2025-09-07 01:07:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.467276362s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.527411741s FAIL (2025-09-07 01:06:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.527446657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.53897223s FAIL (2025-09-07 01:06:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.539065886s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.606379686s FAIL (2025-09-07 01:05:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.606520012s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.401231983s FAIL (2025-09-07 01:03:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.401265135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.274445437s FAIL (2025-09-07 01:02:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.274474693s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.684591959s FAIL (2025-09-07 01:02:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.684625187s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.483256466s FAIL (2025-09-07 01:01:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.483299061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.401904267s FAIL (2025-09-07 01:01:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.401957242s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.387922749s FAIL (2025-09-07 01:00:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.387970999s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.334394944s FAIL (2025-09-07 00:59:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.334424413s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.57218548s FAIL (2025-09-07 00:58:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.572218364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.481134314s FAIL (2025-09-07 00:58:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.481163097s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.446715446s FAIL (2025-09-07 00:57:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.446755137s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.349848194s FAIL (2025-09-07 00:56:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.349878853s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.441914546s FAIL (2025-09-07 00:55:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.441951549s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.115214765s FAIL (2025-09-07 00:55:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 7 00:55:18.487897580 (202h47m38.980330033s) [upload=178411]: Starting upload Sep 7 00:55:18.488017571 (202h47m38.980450017s) [upload=178411]: Done starting upload Sep 7 00:55:18.488187992 (202h47m38.980620447s) [upload=178411]: Uploading segments... Sep 7 00:55:18.488413431 (202h47m38.980845883s) [upload=178411]: Got next segment. Inline: false Sep 7 00:55:18.488481488 (202h47m38.980913941s) [upload=178411]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 00:55:18.488488373 (202h47m38.980920823s) [upload=178411]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 00:55:18.596843838 (202h47m39.089276272s) [seg_pos={0 0}, upload=178411]: Begin upload segment... Sep 7 00:55:18.596904951 (202h47m39.089337384s) [seg_pos={0 0}, upload=178411]: Done begin upload segment. Sep 7 00:55:18.596963777 (202h47m39.089396224s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.597566788 (202h47m39.089999256s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598203343 (202h47m39.090635796s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598620236 (202h47m39.091052689s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.598998012 (202h47m39.091430456s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599334656 (202h47m39.091767102s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.599804440 (202h47m39.092236884s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.600087372 (202h47m39.092519816s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602051967 (202h47m39.094484421s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.602710162 (202h47m39.095142615s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.603278361 (202h47m39.095710815s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.604584178 (202h47m39.097016648s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.605700919 (202h47m39.098133365s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.606751932 (202h47m39.09918439s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.607218822 (202h47m39.099651284s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:18.610013014 (202h47m39.102445475s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Uploading piece... Sep 7 00:55:19.209547411 (202h47m39.701979846s) [upload=178411]: Next returned nil segment Sep 7 00:55:19.209565889 (202h47m39.701998333s) [upload=178411]: Waiting for error group managing segments... Sep 7 00:55:19.350466526 (202h47m39.842898962s) [noise=true , address=45.91.116.4:4937 , piece_id=UFQLAWDPX3RBCTBDPYM6JGYC2DHSNY4IJH5CSBDKVJHXVIBMOGVA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.350552317 (202h47m39.84298475s) [noise=true , address=45.91.116.4:4940 , piece_id=NEUH7JPDWEXYZQGHLTKJ7S4LZQXBV4QXWV7MYUXBTW7BAN66CZBQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.357993088 (202h47m39.850425521s) [noise=true , address=45.91.116.4:4938 , piece_id=7LXFV24LO6CRE3YOKC52UFQQXZZJTMRBY4IO3YWZXDL62374OJ6A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.358432521 (202h47m39.85086495s) [noise=true , address=45.91.116.8:4047 , piece_id=TI375EZ4OCYT5NWEF3R7KLGBQBAAFH3GCV2INUF56NP4PHUDFTIA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.359649148 (202h47m39.852081583s) [noise=true , address=45.91.116.11:4117 , piece_id=L6C5RDKM3G2L2ULVURKULHRSHVL5YUKDURG5UHMLCRPF2INSFLHA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.361849215 (202h47m39.85428166s) [noise=true , address=45.91.116.11:4119 , piece_id=YQXRCGCMQAXGPRUHNLJO6EPHUPYBHIMRKNQBO275PTWH4B3746EA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.374002878 (202h47m39.866435314s) [noise=true , address=45.91.116.10:4082 , piece_id=2DOOBXIP3B2MPCOWFF3TN4N55DE2MMZQS4OPDMFS2ZNISBXMPREQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.386486834 (202h47m39.878919267s) [noise=true , address=45.91.116.7:4009 , piece_id=USRINOQJIGTC5WNNP2MRWYAR4AKUOSMI6OLSTVN4JSB4PGGOBHSA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.430521842 (202h47m39.922954282s) [noise=true , address=77.90.31.10:4517 , piece_id=TUFIBQU3LC5QXIF35NC642HZAV5LNDQYX3ZZ6BH7E2YU4WKGIPKA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.434786954 (202h47m39.927219382s) [noise=true , address=77.90.31.10:4520 , piece_id=KRMDXQKIOLKAGEYJFKOUIOPEO7O5SCVEAWR5J2T3VGK3R3JSFDZA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:19.482194608 (202h47m39.974627046s) [noise=true , address=45.91.116.13:4154 , piece_id=PIRVAGMONYVJAUFE2NU3AXF6YIVMGFWKYU3N22Y6CH54FTKLDXKQ, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:20.266235033 (202h47m40.758667481s) [noise=true , address=45.91.116.12:4226 , piece_id=2ADC6MRVLT3PYWC2V6RXWG7HG3663BNCY6U4NHRKVAKXPTRSHQPA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: <nil> Sep 7 00:55:48.205494295 (202h48m8.69792673s) [noise=true , address=77.90.31.19:4842 , piece_id=WGVGUOSU74P764A3TBUY7IB47VJB2AVZB6XVLWZOHPTNDDF65I3A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205631471 (202h48m8.698063915s) [noise=true , address=77.90.31.19:4841 , piece_id=STKYV56UZZUJYOT2SW5O6DJR6XP6YKDKVAMS47E2JL2Y25LQQQ2Q, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205639627 (202h48m8.698072061s) [noise=true , address=77.90.31.13:4626 , piece_id=NESYO34EJMEVNT2MBOKGPQPHFYFEDWBOFXZV5PJRDVKDSLLLSR2A, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: expected piece hash; context deadline exceeded; EOF Sep 7 00:55:48.205711973 (202h48m8.698144416s) [noise=true , address=77.90.31.19:4843 , piece_id=MT2CNC6LUIYBLS77ZH2FWXKTTKRU6GKXSIROCH72SXX22KEZHWCA, seg_pos={0 0}, upload=178411]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 7 00:55:48.205744748 (202h48m8.698177199s) [seg_pos={0 0}, upload=178411]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 7 00:55:48.318412621 (202h48m8.81084506s) [upload=178411]: Done uploading segments... Sep 7 00:55:48.318451539 (202h48m8.81088398s) [upload=178411]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.115270837s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.771889717s FAIL (2025-09-06 16:40:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.771911974s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.714845429s FAIL (2025-09-06 16:39:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.714870766s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.901446231s FAIL (2025-09-06 16:38:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.901530355s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.879700079s FAIL (2025-09-06 16:37:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.8801956s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.784021865s FAIL (2025-09-06 16:36:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.784048369s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.83440467s FAIL (2025-09-06 16:36:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.834430628s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.904354147s FAIL (2025-09-06 16:35:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.904375503s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 3.000140458s FAIL (2025-09-06 16:35:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.000167572s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.047431943s FAIL (2025-09-06 16:34:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.04751969s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)