Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.457658358s FAIL (2025-07-18 10:55:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.457689319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.752510086s FAIL (2025-07-18 10:55:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.752566941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.359899629s FAIL (2025-07-18 10:54:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.35993265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.573469964s FAIL (2025-07-18 10:54:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.573504783s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.413142532s FAIL (2025-07-18 10:53:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.413193916s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.542523179s FAIL (2025-07-18 10:53:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.542570026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.423420173s FAIL (2025-07-18 10:52:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.423447725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 3.16752541s FAIL (2025-07-18 10:52:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.167554947s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.514451131s FAIL (2025-07-18 10:52:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.514494793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.648763377s FAIL (2025-07-18 10:51:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.648794167s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.306865351s FAIL (2025-07-18 10:51:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.306907721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.995746352s FAIL (2025-07-18 10:50:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.995808287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.455341679s FAIL (2025-07-18 10:50:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.455440587s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.501500978s FAIL (2025-07-18 10:49:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.501541851s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.017203545s FAIL (2025-07-18 10:49:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.01724063s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.546658475s FAIL (2025-07-18 10:49:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.546691558s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.72827198s FAIL (2025-07-18 10:48:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.728312091s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.76550214s FAIL (2025-07-18 10:48:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.765597353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.374953913s FAIL (2025-07-18 10:47:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.374990332s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.482318116s FAIL (2025-07-18 10:47:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.482338593s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 3.347374163s FAIL (2025-07-18 10:47:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.347422822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.382360362s FAIL (2025-07-18 10:46:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.382393691s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.25492558s FAIL (2025-07-18 10:46:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.254954034s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.676663049s FAIL (2025-07-18 10:46:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.676694173s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.448946873s FAIL (2025-07-18 10:45:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.448986317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.589319347s FAIL (2025-07-18 10:45:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.589348575s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.951711256s FAIL (2025-07-18 10:44:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.951795516s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.302078884s FAIL (2025-07-18 10:44:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.302111554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.96419816s FAIL (2025-07-18 10:44:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.964224346s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.344011207s FAIL (2025-07-18 10:43:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.344045115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.553672143s FAIL (2025-07-18 10:43:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.553711201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 3.522747031s FAIL (2025-07-18 10:42:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 3.52282758s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.512693773s FAIL (2025-07-18 10:42:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.512753645s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.337736919s FAIL (2025-07-18 10:42:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.337778761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.544839657s FAIL (2025-07-18 10:41:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.544873043s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.319037147s FAIL (2025-07-18 10:41:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.319072102s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.617354487s FAIL (2025-07-18 10:40:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.617385865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.37402644s FAIL (2025-07-18 10:40:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.374050653s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.45715962s FAIL (2025-07-18 10:40:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.4571995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.737416798s FAIL (2025-07-18 10:39:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.737455116s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.472481556s FAIL (2025-07-18 10:39:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.472510351s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.634801085s FAIL (2025-07-18 10:39:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.634831111s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.674777276s FAIL (2025-07-18 10:38:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.67490426s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.508853015s FAIL (2025-07-18 10:38:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.508885088s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.358515409s FAIL (2025-07-18 10:38:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.358599623s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.695443715s FAIL (2025-07-18 10:37:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.695542632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.345119716s FAIL (2025-07-18 10:37:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.345164968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.353245329s FAIL (2025-07-18 10:37:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.35327811s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.701282438s FAIL (2025-07-18 10:37:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.701337839s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.295428925s FAIL (2025-07-18 10:36:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.295457286s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)