Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.526611694s FAIL (2025-07-18 10:36:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.52664883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.362485938s FAIL (2025-07-18 10:35:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.362518089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.808894187s FAIL (2025-07-18 10:35:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.808927729s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.508276285s FAIL (2025-07-18 10:34:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.508310343s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.416439562s FAIL (2025-07-18 10:34:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.416471317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.820720706s FAIL (2025-07-18 10:33:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.82076668s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.359055638s FAIL (2025-07-18 10:33:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.359086285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.802324732s FAIL (2025-07-18 10:33:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.802357466s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.647832935s FAIL (2025-07-18 10:32:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.647861326s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X........................................X..................] duration: 2.41829505s FAIL (2025-07-18 10:32:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.418326343s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.599071846s FAIL (2025-07-18 10:32:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.599095999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X........................................X.................] duration: 2.398346185s FAIL (2025-07-18 10:31:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.398378383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.720579618s FAIL (2025-07-18 10:31:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.720667978s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X........................................X................] duration: 2.346944602s FAIL (2025-07-18 10:31:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.347155705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.739324952s FAIL (2025-07-18 10:31:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.739356322s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X........................................X...............] duration: 2.315002759s FAIL (2025-07-18 10:30:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.315037472s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.839864091s FAIL (2025-07-18 10:30:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.839894517s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X........................................X..............] duration: 2.330507689s FAIL (2025-07-18 10:29:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.330538491s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.424194542s FAIL (2025-07-18 10:29:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.424227305s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X........................................X.............] duration: 2.345642059s FAIL (2025-07-18 10:28:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.345668672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.479511193s FAIL (2025-07-18 10:28:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.479545108s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.68407455s FAIL (2025-07-18 10:28:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.684109092s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X........................................X............] duration: 2.477616399s FAIL (2025-07-18 10:27:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.47764891s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X........................................X...........] duration: 2.413908681s FAIL (2025-07-18 10:27:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.413940453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.66034426s FAIL (2025-07-18 10:27:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.660374285s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X........................................X..........] duration: 2.552456757s FAIL (2025-07-18 10:26:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.552487937s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.5457863s FAIL (2025-07-18 10:26:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.545827411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X........................................X.........] duration: 2.47462201s FAIL (2025-07-18 10:25:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.474652252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.522711407s FAIL (2025-07-18 10:25:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.522742691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X........................................X........] duration: 2.554513834s FAIL (2025-07-18 10:24:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.554543844s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.974330523s FAIL (2025-07-18 10:24:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.974365982s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X........................................X.......] duration: 2.435503212s FAIL (2025-07-18 10:24:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.435531841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.753563331s FAIL (2025-07-18 10:23:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.753600188s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X........................................X......] duration: 4.035189875s FAIL (2025-07-18 10:22:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 4.035227691s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.904239395s FAIL (2025-07-18 10:22:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.904275807s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X........................................X.....] duration: 2.444362246s FAIL (2025-07-18 10:22:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.444400162s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X........................................X....] duration: 2.355596343s FAIL (2025-07-18 10:22:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.355626004s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.484925097s FAIL (2025-07-18 10:22:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.484958036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X........................................X...] duration: 2.389796726s FAIL (2025-07-18 10:21:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.389831621s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X........................................X..] duration: 2.383313756s FAIL (2025-07-18 10:21:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.383387356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.958935795s FAIL (2025-07-18 10:21:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.959051409s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X........................................X.] duration: 2.317542558s FAIL (2025-07-18 10:20:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.317570622s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X........................................X] duration: 30.126647314s FAIL (2025-07-18 10:20:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 480650]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 10:18:47.533942446 (601h7m47.417356079s) [upload=480650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 10:18:47.812231852 (601h7m47.695645478s) [upload=480650]: Done uploading segments... Jul 18 10:18:47.812262386 (601h7m47.695676006s) [upload=480650]: Upload finished. err: <nil> Jul 18 10:19:55.918921928 (601h8m55.802335557s) [upload=480662]: Starting upload Jul 18 10:19:55.919029214 (601h8m55.802442841s) [upload=480662]: Done starting upload Jul 18 10:19:55.919155429 (601h8m55.802569051s) [upload=480662]: Uploading segments... Jul 18 10:19:55.919515206 (601h8m55.802928835s) [upload=480662]: Got next segment. Inline: false Jul 18 10:19:55.919571748 (601h8m55.802985369s) [upload=480662]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:19:55.919578101 (601h8m55.802991727s) [upload=480662]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:19:56.031248080 (601h8m55.914661696s) [seg_pos={0 0}, upload=480662]: Begin upload segment... Jul 18 10:19:56.031292240 (601h8m55.914705855s) [seg_pos={0 0}, upload=480662]: Done begin upload segment. Jul 18 10:19:56.031317268 (601h8m55.914730898s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031351551 (601h8m55.914765183s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031690493 (601h8m55.915104123s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031812851 (601h8m55.915226478s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.031989185 (601h8m55.915402815s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032350596 (601h8m55.915764228s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032484269 (601h8m55.915897899s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032667765 (601h8m55.916081392s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032790153 (601h8m55.916203782s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.032975878 (601h8m55.916389503s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033092815 (601h8m55.916506452s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033265110 (601h8m55.916678735s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033381530 (601h8m55.91679516s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033556914 (601h8m55.91697054s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.033711089 (601h8m55.917124725s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.034072226 (601h8m55.91748586s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Uploading piece... Jul 18 10:19:56.636472896 (601h8m56.519886527s) [upload=480662]: Next returned nil segment Jul 18 10:19:56.636496664 (601h8m56.51991028s) [upload=480662]: Waiting for error group managing segments... Jul 18 10:19:56.771748829 (601h8m56.655162461s) [noise=true , address=77.90.31.18:4806 , piece_id=NO4NHTDSAH4UUNLH4JF4WVJOVTH5YYN3MLXH6MJZLB5L7VPTN36A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.774291101 (601h8m56.657704713s) [noise=true , address=77.90.31.14:4662 , piece_id=PQEQRAMXQ6ISPUEGVC7H3QBW4RGELHSQRVGZ73ZN7S2MYIK4DK2Q, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.777786273 (601h8m56.661199903s) [noise=true , address=45.91.116.6:4973 , piece_id=HP3EHZ7DTMK5NROFME4BB6L35GNHP4BUZ7GXMMMHNQNE5QIBQNRA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.792902845 (601h8m56.676316463s) [noise=true , address=45.91.116.8:4048 , piece_id=CGRSYOZRYATMSMSUFQOJWVCRXSTH4XNIHOPLHB2SO3LKPA7VYHUA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.793108817 (601h8m56.676522435s) [noise=true , address=45.91.116.11:4117 , piece_id=JQEYTY2QX7PEGBA2HB3S3YHBN6BLM7JJ7ZMRROC6RPTTHKVZGPPA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.798027666 (601h8m56.6814413s) [noise=true , address=77.90.31.19:4842 , piece_id=C3PIB4BNPDUDFVCE7CUBBCJEJVXMST2HG5FRYN4GMKQNEGMKZQCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.801662815 (601h8m56.685076444s) [noise=true , address=104.36.82.15:4257 , piece_id=ZLRHK52WBIITKAN7XL3IXRUXARJZHG4MBYCNKQKAEDZM3LO4SZCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.807815370 (601h8m56.691229003s) [noise=true , address=104.36.82.13:4186 , piece_id=PKBR6JU3XOQDZVW2L6XSE3PBJCYNIHJFF7QAZGU5L32ATPCT2WQA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.816684873 (601h8m56.700098485s) [noise=true , address=104.36.82.2:4731 , piece_id=QL3J2KZ7JSL6XCT7BEBTICYDNO7J46BKMFWGVZGHFXHL4HJCJMCQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:19:56.822376971 (601h8m56.705790598s) [noise=true , address=104.36.82.14:4222 , piece_id=UEDV5F73GO5VVWRS6OUFFU4DC5KH2E7J6ORUAS6ENH6X3JASIOSQ, seg_pos={0 0}, upload=480662]: Done uploading piece. err: <nil> Jul 18 10:20:25.646275878 (601h9m25.529689504s) [noise=true , address=77.90.31.11:4554 , piece_id=TMJCTAKQSBU43GAPDKFRO4CV25F3NUSXLRR3BHJYWGJHLAH6U2GA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: rpc: dial tcp 77.90.31.11:4554: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646361211 (601h9m25.529774832s) [noise=true , address=77.90.31.13:4627 , piece_id=ET6E4UJJY4BED3BF6OMIR7M3FUUQI2HS6EXBETDW5VZ7A4TKY63A, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:40210->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646422784 (601h9m25.529836403s) [noise=true , address=77.90.31.17:4771 , piece_id=DTFPLMPZCBZAEXQQVIHFAXYDCPQLV7RTSLTG5LZ3R4JWW6VFOIFA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 139.59.144.22:54416->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646498682 (601h9m25.529912301s) [noise=true , address=77.90.31.16:4734 , piece_id=OZOFFS3ATHKYAQX5IF3N7ZLBFEIRPPCQKFWJBGPSSB4PL4ZKGYLA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 139.59.144.22:35464->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646512514 (601h9m25.529926133s) [noise=true , address=77.90.31.19:4843 , piece_id=35FVX7VL3RSJJSJYB6B63XDL7KBURU6LKWDE2EUOM4YHX477DQHA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646569038 (601h9m25.529982655s) [noise=true , address=77.90.31.10:4519 , piece_id=BGSDLEI6DGABQ6MPE6MKB3GQTSZFCLU2SMHXOYOTJAXIPOPPQUMA, seg_pos={0 0}, upload=480662]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:20:25.646604894 (601h9m25.530018531s) [seg_pos={0 0}, upload=480662]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 18 10:20:25.770785641 (601h9m25.654199264s) [upload=480662]: Done uploading segments... Jul 18 10:20:25.770811727 (601h9m25.654225346s) [upload=480662]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.126675941s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.060380526s FAIL (2025-07-18 10:20:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 3.060423016s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.651392008s FAIL (2025-07-18 10:19:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.651423618s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.557922493s FAIL (2025-07-18 10:19:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.557956079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.446950199s FAIL (2025-07-18 10:18:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eg_pos={0 0}, upload=480208]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Jul 18 09:46:29.852244705 (600h35m29.735658318s) [seg_pos={0 0}, upload=480208]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 09:46:29.852337980 (600h35m29.735751616s) [upload=480208]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 09:46:29.852343363 (600h35m29.735757s) [upload=480208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 09:46:30.018458479 (600h35m29.901872113s) [upload=480208]: Done uploading segments... Jul 18 09:46:30.018482701 (600h35m29.901896329s) [upload=480208]: Upload finished. err: <nil> Jul 18 09:47:12.249795116 (600h36m12.133208733s) [upload=480219]: Starting upload Jul 18 09:47:12.249858020 (600h36m12.133271649s) [upload=480219]: Done starting upload Jul 18 09:47:12.249967722 (600h36m12.133381349s) [upload=480219]: Uploading segments... Jul 18 09:47:12.250258050 (600h36m12.133671676s) [upload=480219]: Got next segment. Inline: false Jul 18 09:47:12.250305440 (600h36m12.133719067s) [upload=480219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 09:47:12.250311644 (600h36m12.13372527s) [upload=480219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 09:47:12.360334967 (600h36m12.243748582s) [seg_pos={0 0}, upload=480219]: Begin upload segment... Jul 18 09:47:12.360398232 (600h36m12.243811858s) [seg_pos={0 0}, upload=480219]: Done begin upload segment. Jul 18 09:47:12.360430446 (600h36m12.243844075s) [noise=true , address=45.91.116.4:4940 , piece_id=6VYCYZAAU43AU43LELYGEF43C6PETFOPDPYAWA4M4LN46I2SYWXA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360442158 (600h36m12.243855784s) [noise=true , address=77.90.31.17:4772 , piece_id=MNJSLMK3AYBBCNNGB2XCEJVK6GOZON5XNFTSAJ7WZFDVDLYYHMYA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360844768 (600h36m12.244258405s) [noise=true , address=104.36.82.4:4803 , piece_id=WSVRUI2UH7HYBGKVSPJQWFKTTBKY4C2RGLS532ECHQXAYBIKGIRA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360921527 (600h36m12.244335154s) [noise=true , address=45.91.116.14:4190 , piece_id=I3NIKZIQUGPLUGRUB52UTINRQVTXED2OMZKF5ZKK2PUIEA2OKXTQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361231814 (600h36m12.244645447s) [noise=true , address=45.91.116.4:4937 , piece_id=PDRF44GF42EBXT5WGJU2FWPW2PIIAJ7BQDYMYPICE4TPHOIMVRNQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361272926 (600h36m12.244686561s) [noise=true , address=77.90.31.13:4628 , piece_id=6MQPSFO2NC3WUHLT3EV2P2MTQ6662V6RLFMDJNWJODEWUKKCDKPA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361573071 (600h36m12.244986705s) [noise=true , address=77.90.31.14:4661 , piece_id=KOZPREPIRIJ33RXIVT3MELOHCLF3BNK52K43J2TYD6NUF27SZRFQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362070831 (600h36m12.245484467s) [noise=true , address=77.90.31.16:4734 , piece_id=TMPHZML4KOSCYFBC7S7CYAL2MFXF77YKOSFW73UA7QBU4ZI4VQNA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362578404 (600h36m12.24599204s) [noise=true , address=77.90.31.21:4877 , piece_id=GO3ASSKXAGKOVQAGR2C5QSO7ZZHD53PXNILFEZ2ZIMAEG5E6CZNQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362639615 (600h36m12.24605325s) [noise=true , address=45.91.116.9:4263 , piece_id=Y6JBE25Y5FIJJZMSCOGKMWGSKJGQ7CQNETGEOJTGCTB3K45VIAZQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.363118673 (600h36m12.246532308s) [noise=true , address=77.90.31.10:4519 , piece_id=HQMJ62CTMCVMDMOMZKVBJVOXD3FAYBQ5ZUWUUB22OZKZDFOGNJ3Q, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.363610806 (600h36m12.247024439s) [noise=true , address=45.91.116.11:4118 , piece_id=PCHVHKJYKVF3RO3LCNUXFCEHKBNNLQBBRZISAHGKHRAEQQ4BRJVQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364064565 (600h36m12.247478194s) [noise=true , address=45.91.116.13:4153 , piece_id=M2FJOC4YSD5YU6NUKQMFTDLEVVCGSFXD42UBFSWFKPDK6RQKPMBA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364390745 (600h36m12.247804371s) [noise=true , address=77.90.31.18:4806 , piece_id=I6HTRAOTH5CKEY6POVTA3RMVMKXIFTUAGCKRF32W7YSTULSOKKBQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364738064 (600h36m12.248151694s) [noise=true , address=45.91.116.6:4974 , piece_id=5AD6RWWSC6PAGM7CCGYA6LVOY4EHIAZ23RNIG6232NUC3MNWCA6A, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.365227530 (600h36m12.248641159s) [noise=true , address=77.90.31.13:4627 , piece_id=VE4A2OPBOCZKXAEOKEQBXEKAXH4MMUVWUC7G4B4FUF26O3MS3VRA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.977845102 (600h36m12.861258716s) [upload=480219]: Next returned nil segment Jul 18 09:47:12.977863037 (600h36m12.86127665s) [upload=480219]: Waiting for error group managing segments... Jul 18 09:47:13.101023422 (600h36m12.984437052s) [noise=true , address=77.90.31.17:4772 , piece_id=MNJSLMK3AYBBCNNGB2XCEJVK6GOZON5XNFTSAJ7WZFDVDLYYHMYA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.112120257 (600h36m12.99553389s) [noise=true , address=45.91.116.4:4937 , piece_id=PDRF44GF42EBXT5WGJU2FWPW2PIIAJ7BQDYMYPICE4TPHOIMVRNQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.121688728 (600h36m13.005102347s) [noise=true , address=45.91.116.4:4940 , piece_id=6VYCYZAAU43AU43LELYGEF43C6PETFOPDPYAWA4M4LN46I2SYWXA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.121982112 (600h36m13.005395726s) [noise=true , address=45.91.116.11:4118 , piece_id=PCHVHKJYKVF3RO3LCNUXFCEHKBNNLQBBRZISAHGKHRAEQQ4BRJVQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.123111038 (600h36m13.006524658s) [noise=true , address=77.90.31.16:4734 , piece_id=TMPHZML4KOSCYFBC7S7CYAL2MFXF77YKOSFW73UA7QBU4ZI4VQNA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.123623025 (600h36m13.007036645s) [noise=true , address=45.91.116.13:4153 , piece_id=M2FJOC4YSD5YU6NUKQMFTDLEVVCGSFXD42UBFSWFKPDK6RQKPMBA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.124578753 (600h36m13.007992377s) [noise=true , address=45.91.116.6:4974 , piece_id=5AD6RWWSC6PAGM7CCGYA6LVOY4EHIAZ23RNIG6232NUC3MNWCA6A, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.128637729 (600h36m13.012051342s) [noise=true , address=45.91.116.14:4190 , piece_id=I3NIKZIQUGPLUGRUB52UTINRQVTXED2OMZKF5ZKK2PUIEA2OKXTQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.131102604 (600h36m13.014516217s) [noise=true , address=45.91.116.9:4263 , piece_id=Y6JBE25Y5FIJJZMSCOGKMWGSKJGQ7CQNETGEOJTGCTB3K45VIAZQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.132579066 (600h36m13.015992698s) [noise=true , address=77.90.31.18:4806 , piece_id=I6HTRAOTH5CKEY6POVTA3RMVMKXIFTUAGCKRF32W7YSTULSOKKBQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.142520333 (600h36m13.025933952s) [noise=true , address=77.90.31.13:4628 , piece_id=6MQPSFO2NC3WUHLT3EV2P2MTQ6662V6RLFMDJNWJODEWUKKCDKPA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.158118396 (600h36m13.041532015s) [noise=true , address=104.36.82.4:4803 , piece_id=WSVRUI2UH7HYBGKVSPJQWFKTTBKY4C2RGLS532ECHQXAYBIKGIRA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:41.976241305 (600h36m41.859654924s) [noise=true , address=77.90.31.13:4627 , piece_id=VE4A2OPBOCZKXAEOKEQBXEKAXH4MMUVWUC7G4B4FUF26O3MS3VRA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:48966->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976424161 (600h36m41.859837775s) [noise=true , address=77.90.31.21:4877 , piece_id=GO3ASSKXAGKOVQAGR2C5QSO7ZZHD53PXNILFEZ2ZIMAEG5E6CZNQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976474506 (600h36m41.859888139s) [noise=true , address=77.90.31.10:4519 , piece_id=HQMJ62CTMCVMDMOMZKVBJVOXD3FAYBQ5ZUWUUB22OZKZDFOGNJ3Q, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 139.59.144.22:40678->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976493548 (600h36m41.859907161s) [noise=true , address=77.90.31.14:4661 , piece_id=KOZPREPIRIJ33RXIVT3MELOHCLF3BNK52K43J2TYD6NUF27SZRFQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976522792 (600h36m41.859936418s) [seg_pos={0 0}, upload=480219]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 09:47:42.089123138 (600h36m41.972536754s) [upload=480219]: Done uploading segments... Jul 18 09:47:42.089150713 (600h36m41.972564329s) [upload=480219]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.446986445s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.766593998s FAIL (2025-07-18 10:18:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.766628415s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.77346725s FAIL (2025-07-18 10:17:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 599]: Uploading segments... Jul 18 10:15:11.294485085 (601h4m11.177898712s) [upload=480599]: Got next segment. Inline: false Jul 18 10:15:11.294544272 (601h4m11.177957885s) [upload=480599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 10:15:11.294550582 (601h4m11.177964195s) [upload=480599]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 10:15:11.501406888 (601h4m11.384820518s) [seg_pos={0 0}, upload=480599]: Begin upload segment... Jul 18 10:15:11.501484361 (601h4m11.384897977s) [seg_pos={0 0}, upload=480599]: Done begin upload segment. Jul 18 10:15:11.501520374 (601h4m11.384934003s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.501569822 (601h4m11.384983455s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502348936 (601h4m11.385762573s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.502573055 (601h4m11.38598669s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503277563 (601h4m11.386691202s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.503364061 (601h4m11.386777691s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504187187 (601h4m11.387600818s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504334443 (601h4m11.387748073s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.504879053 (601h4m11.388292692s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505024311 (601h4m11.38843794s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505615030 (601h4m11.389028658s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.505818478 (601h4m11.389232111s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506194275 (601h4m11.389607905s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.506684236 (601h4m11.390097863s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507042490 (601h4m11.390456119s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507352345 (601h4m11.390765974s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507641444 (601h4m11.391055073s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:11.507920530 (601h4m11.391334158s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Uploading piece... Jul 18 10:15:12.073767728 (601h4m11.957181359s) [upload=480599]: Next returned nil segment Jul 18 10:15:12.073789153 (601h4m11.95720279s) [upload=480599]: Waiting for error group managing segments... Jul 18 10:15:12.220745306 (601h4m12.104158944s) [noise=true , address=45.91.116.13:4165 , piece_id=HTDTXYPQCHJ44P6TZK5NHBAA6VLQ3ZAPGICXWNAIY7XTAJFPXFLQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.221065650 (601h4m12.104479281s) [noise=true , address=45.91.116.9:4290 , piece_id=V577LFCRNEMU5O2MVJIWZFCQKEP73OPTSYTJGURWU7LGCNR5A3MQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.223269203 (601h4m12.10668284s) [noise=true , address=45.91.116.14:4202 , piece_id=G66FPECBOW7U6THNNJQ7C5FGSFI6HRSNUPSK2QABRCHNBHAFJMVQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.225214965 (601h4m12.108628586s) [noise=true , address=77.90.31.18:4817 , piece_id=RZDH6XKPAJLMA5IVO6MYRKZRKFVA43CY6QMLV3Y7AXA7ZL6PBXBA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.233054290 (601h4m12.116467916s) [noise=true , address=45.91.116.11:4134 , piece_id=6GOQ3JZ2LO6TTXTEQRGGDPRESM5WN7VYHJ6I64EBD7DSLB5WFINQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.236960016 (601h4m12.12037364s) [noise=true , address=45.91.116.7:4018 , piece_id=5VSJRU2YLNKO4N7HN3E6XQAI5MUAPTJESGYPQH2WYFK6SXM4TNOQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.240699624 (601h4m12.12411326s) [noise=true , address=77.90.31.14:4681 , piece_id=AAVMUTZDZPL4OAUFWYBDVVEOB4QD6QXVUU3TJFJTQRXXNYIXO5UQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.241728084 (601h4m12.125141702s) [noise=true , address=45.91.116.4:4961 , piece_id=SNGSTP247D5AOHP3XSZT2STVOSQODBBG527SZSM2UFIZT6AN7ZZQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.244959034 (601h4m12.128372651s) [noise=true , address=77.90.31.19:4869 , piece_id=X3RQKYZNYVNQ6FQPEJWF7JSP73YCZOZCNI2BZAUODXBUWY6JM23Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.374213489 (601h4m12.257627126s) [noise=true , address=45.91.95.2:4029 , piece_id=NVUFZDB5CC72ITPBTYUXI67ZCB37YYEHQQDFKD4M4FH47OYPI6OQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.388697463 (601h4m12.272111089s) [noise=true , address=79.127.220.97:4384 , piece_id=YTWOECX23IN7XDUXTUIMJ2LWT2SVKHIBFJMJ4DUXF73WR3USMO4Q, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.461880149 (601h4m12.345293763s) [noise=true , address=45.91.95.4:4034 , piece_id=LUDQUXHNRGSIYJKGIOPWDAXYPQNPWCJ2CRXAJVTEAJJVGSSYP6DA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:15:12.743207026 (601h4m12.626620643s) [noise=true , address=79.127.205.235:4690 , piece_id=KGMPALZUII5C5ZBGFIQ5DUX4D74ZIBLX453AHAEONK6PJCIVZBRA, seg_pos={0 0}, upload=480599]: Done uploading piece. err: <nil> Jul 18 10:16:11.024574729 (601h5m10.907988355s) [noise=true , address=77.90.31.17:4799 , piece_id=LJMMDBRARSETLLBK7CG76OO4EDI3IMQJ77BH2IGDDIEKSWZR24VQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12UhAksgXZuFGAVheaYzP7D4kwVyAjUAq3BtrvdSxszjNCFCTWT, address:77.90.31.17:4799): protocol: rpc: dial tcp 77.90.31.17:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024635598 (601h5m10.908049211s) [noise=true , address=77.90.31.10:4526 , piece_id=KSQNNDUQZ4H2E2LUI2D273BS2ZA7KVL5JLZ26LXNMF2P2PKKVSHQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12NKFSj1P8HBhfYrYPTsj5hANnpp3FZmMkuWvqpS7MxKR3k6Bq, address:77.90.31.10:4526): protocol: write tcp 139.59.144.22:49184->77.90.31.10:4526: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024688821 (601h5m10.908102433s) [noise=true , address=77.90.31.21:4905 , piece_id=V2FOQZSC6BFX32AW5GC2Z6L5ZH6YYWFOBJVZWE6TUSCD3HAITXPQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:1X5A7bLqMEQuaMbf9xZ7PDUD4161sz2QqdM9r59y3bsyPBJhaK, address:77.90.31.21:4905): protocol: rpc: dial tcp 77.90.31.21:4905: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024730241 (601h5m10.908143871s) [noise=true , address=77.90.31.16:4745 , piece_id=O4LBEBW7DJZE436DA4I5N4JLRYICMZELZRJPEFVJTTK3CDO2ZWXQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:12sFKft9P2avVAwrZC4fcCRwuWbFT7q9bCwLmumHNmSiqxHDR7r, address:77.90.31.16:4745): protocol: rpc: dial tcp 77.90.31.16:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024791318 (601h5m10.908204944s) [noise=true , address=77.90.31.15:4728 , piece_id=OZMJCIHTZ6IDSVI7ZCVCNT4H5JLMEGUOK2YNUDPQPXUUTW2SYCNQ, seg_pos={0 0}, upload=480599]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: write tcp 139.59.144.22:45178->77.90.31.15:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 10:16:11.024827506 (601h5m10.908241136s) [seg_pos={0 0}, upload=480599]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 10:16:11.138404604 (601h5m11.02181823s) [upload=480599]: Done uploading segments... Jul 18 10:16:11.138432097 (601h5m11.02184571s) [upload=480599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.773489363s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.451629392s FAIL (2025-07-18 10:17:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eg_pos={0 0}, upload=480208]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Jul 18 09:46:29.852244705 (600h35m29.735658318s) [seg_pos={0 0}, upload=480208]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 09:46:29.852337980 (600h35m29.735751616s) [upload=480208]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 09:46:29.852343363 (600h35m29.735757s) [upload=480208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 09:46:30.018458479 (600h35m29.901872113s) [upload=480208]: Done uploading segments... Jul 18 09:46:30.018482701 (600h35m29.901896329s) [upload=480208]: Upload finished. err: <nil> Jul 18 09:47:12.249795116 (600h36m12.133208733s) [upload=480219]: Starting upload Jul 18 09:47:12.249858020 (600h36m12.133271649s) [upload=480219]: Done starting upload Jul 18 09:47:12.249967722 (600h36m12.133381349s) [upload=480219]: Uploading segments... Jul 18 09:47:12.250258050 (600h36m12.133671676s) [upload=480219]: Got next segment. Inline: false Jul 18 09:47:12.250305440 (600h36m12.133719067s) [upload=480219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 09:47:12.250311644 (600h36m12.13372527s) [upload=480219]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 09:47:12.360334967 (600h36m12.243748582s) [seg_pos={0 0}, upload=480219]: Begin upload segment... Jul 18 09:47:12.360398232 (600h36m12.243811858s) [seg_pos={0 0}, upload=480219]: Done begin upload segment. Jul 18 09:47:12.360430446 (600h36m12.243844075s) [noise=true , address=45.91.116.4:4940 , piece_id=6VYCYZAAU43AU43LELYGEF43C6PETFOPDPYAWA4M4LN46I2SYWXA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360442158 (600h36m12.243855784s) [noise=true , address=77.90.31.17:4772 , piece_id=MNJSLMK3AYBBCNNGB2XCEJVK6GOZON5XNFTSAJ7WZFDVDLYYHMYA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360844768 (600h36m12.244258405s) [noise=true , address=104.36.82.4:4803 , piece_id=WSVRUI2UH7HYBGKVSPJQWFKTTBKY4C2RGLS532ECHQXAYBIKGIRA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.360921527 (600h36m12.244335154s) [noise=true , address=45.91.116.14:4190 , piece_id=I3NIKZIQUGPLUGRUB52UTINRQVTXED2OMZKF5ZKK2PUIEA2OKXTQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361231814 (600h36m12.244645447s) [noise=true , address=45.91.116.4:4937 , piece_id=PDRF44GF42EBXT5WGJU2FWPW2PIIAJ7BQDYMYPICE4TPHOIMVRNQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361272926 (600h36m12.244686561s) [noise=true , address=77.90.31.13:4628 , piece_id=6MQPSFO2NC3WUHLT3EV2P2MTQ6662V6RLFMDJNWJODEWUKKCDKPA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.361573071 (600h36m12.244986705s) [noise=true , address=77.90.31.14:4661 , piece_id=KOZPREPIRIJ33RXIVT3MELOHCLF3BNK52K43J2TYD6NUF27SZRFQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362070831 (600h36m12.245484467s) [noise=true , address=77.90.31.16:4734 , piece_id=TMPHZML4KOSCYFBC7S7CYAL2MFXF77YKOSFW73UA7QBU4ZI4VQNA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362578404 (600h36m12.24599204s) [noise=true , address=77.90.31.21:4877 , piece_id=GO3ASSKXAGKOVQAGR2C5QSO7ZZHD53PXNILFEZ2ZIMAEG5E6CZNQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.362639615 (600h36m12.24605325s) [noise=true , address=45.91.116.9:4263 , piece_id=Y6JBE25Y5FIJJZMSCOGKMWGSKJGQ7CQNETGEOJTGCTB3K45VIAZQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.363118673 (600h36m12.246532308s) [noise=true , address=77.90.31.10:4519 , piece_id=HQMJ62CTMCVMDMOMZKVBJVOXD3FAYBQ5ZUWUUB22OZKZDFOGNJ3Q, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.363610806 (600h36m12.247024439s) [noise=true , address=45.91.116.11:4118 , piece_id=PCHVHKJYKVF3RO3LCNUXFCEHKBNNLQBBRZISAHGKHRAEQQ4BRJVQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364064565 (600h36m12.247478194s) [noise=true , address=45.91.116.13:4153 , piece_id=M2FJOC4YSD5YU6NUKQMFTDLEVVCGSFXD42UBFSWFKPDK6RQKPMBA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364390745 (600h36m12.247804371s) [noise=true , address=77.90.31.18:4806 , piece_id=I6HTRAOTH5CKEY6POVTA3RMVMKXIFTUAGCKRF32W7YSTULSOKKBQ, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.364738064 (600h36m12.248151694s) [noise=true , address=45.91.116.6:4974 , piece_id=5AD6RWWSC6PAGM7CCGYA6LVOY4EHIAZ23RNIG6232NUC3MNWCA6A, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.365227530 (600h36m12.248641159s) [noise=true , address=77.90.31.13:4627 , piece_id=VE4A2OPBOCZKXAEOKEQBXEKAXH4MMUVWUC7G4B4FUF26O3MS3VRA, seg_pos={0 0}, upload=480219]: Uploading piece... Jul 18 09:47:12.977845102 (600h36m12.861258716s) [upload=480219]: Next returned nil segment Jul 18 09:47:12.977863037 (600h36m12.86127665s) [upload=480219]: Waiting for error group managing segments... Jul 18 09:47:13.101023422 (600h36m12.984437052s) [noise=true , address=77.90.31.17:4772 , piece_id=MNJSLMK3AYBBCNNGB2XCEJVK6GOZON5XNFTSAJ7WZFDVDLYYHMYA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.112120257 (600h36m12.99553389s) [noise=true , address=45.91.116.4:4937 , piece_id=PDRF44GF42EBXT5WGJU2FWPW2PIIAJ7BQDYMYPICE4TPHOIMVRNQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.121688728 (600h36m13.005102347s) [noise=true , address=45.91.116.4:4940 , piece_id=6VYCYZAAU43AU43LELYGEF43C6PETFOPDPYAWA4M4LN46I2SYWXA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.121982112 (600h36m13.005395726s) [noise=true , address=45.91.116.11:4118 , piece_id=PCHVHKJYKVF3RO3LCNUXFCEHKBNNLQBBRZISAHGKHRAEQQ4BRJVQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.123111038 (600h36m13.006524658s) [noise=true , address=77.90.31.16:4734 , piece_id=TMPHZML4KOSCYFBC7S7CYAL2MFXF77YKOSFW73UA7QBU4ZI4VQNA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.123623025 (600h36m13.007036645s) [noise=true , address=45.91.116.13:4153 , piece_id=M2FJOC4YSD5YU6NUKQMFTDLEVVCGSFXD42UBFSWFKPDK6RQKPMBA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.124578753 (600h36m13.007992377s) [noise=true , address=45.91.116.6:4974 , piece_id=5AD6RWWSC6PAGM7CCGYA6LVOY4EHIAZ23RNIG6232NUC3MNWCA6A, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.128637729 (600h36m13.012051342s) [noise=true , address=45.91.116.14:4190 , piece_id=I3NIKZIQUGPLUGRUB52UTINRQVTXED2OMZKF5ZKK2PUIEA2OKXTQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.131102604 (600h36m13.014516217s) [noise=true , address=45.91.116.9:4263 , piece_id=Y6JBE25Y5FIJJZMSCOGKMWGSKJGQ7CQNETGEOJTGCTB3K45VIAZQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.132579066 (600h36m13.015992698s) [noise=true , address=77.90.31.18:4806 , piece_id=I6HTRAOTH5CKEY6POVTA3RMVMKXIFTUAGCKRF32W7YSTULSOKKBQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.142520333 (600h36m13.025933952s) [noise=true , address=77.90.31.13:4628 , piece_id=6MQPSFO2NC3WUHLT3EV2P2MTQ6662V6RLFMDJNWJODEWUKKCDKPA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:13.158118396 (600h36m13.041532015s) [noise=true , address=104.36.82.4:4803 , piece_id=WSVRUI2UH7HYBGKVSPJQWFKTTBKY4C2RGLS532ECHQXAYBIKGIRA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: <nil> Jul 18 09:47:41.976241305 (600h36m41.859654924s) [noise=true , address=77.90.31.13:4627 , piece_id=VE4A2OPBOCZKXAEOKEQBXEKAXH4MMUVWUC7G4B4FUF26O3MS3VRA, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 139.59.144.22:48966->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976424161 (600h36m41.859837775s) [noise=true , address=77.90.31.21:4877 , piece_id=GO3ASSKXAGKOVQAGR2C5QSO7ZZHD53PXNILFEZ2ZIMAEG5E6CZNQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976474506 (600h36m41.859888139s) [noise=true , address=77.90.31.10:4519 , piece_id=HQMJ62CTMCVMDMOMZKVBJVOXD3FAYBQ5ZUWUUB22OZKZDFOGNJ3Q, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 139.59.144.22:40678->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976493548 (600h36m41.859907161s) [noise=true , address=77.90.31.14:4661 , piece_id=KOZPREPIRIJ33RXIVT3MELOHCLF3BNK52K43J2TYD6NUF27SZRFQ, seg_pos={0 0}, upload=480219]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 09:47:41.976522792 (600h36m41.859936418s) [seg_pos={0 0}, upload=480219]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 09:47:42.089123138 (600h36m41.972536754s) [upload=480219]: Done uploading segments... Jul 18 09:47:42.089150713 (600h36m41.972564329s) [upload=480219]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.45173458s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)