Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.397153551s FAIL (2025-09-23 21:37:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.397186983s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.550622952s FAIL (2025-09-23 21:36:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.550656145s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.617926734s FAIL (2025-09-23 21:35:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.617957911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.587882792s FAIL (2025-09-23 21:34:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.587913641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.707606694s FAIL (2025-09-23 21:34:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.707639966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.602345867s FAIL (2025-09-23 21:33:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.602386115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.571389163s FAIL (2025-09-23 21:32:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.571425411s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.595040497s FAIL (2025-09-23 21:31:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.595074s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.567290453s FAIL (2025-09-23 21:30:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.567325467s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.503556911s FAIL (2025-09-23 21:29:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.503587813s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.483150792s FAIL (2025-09-23 21:28:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.483179032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.535519483s FAIL (2025-09-23 21:27:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.535550836s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.610312571s FAIL (2025-09-23 21:27:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.610431331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.67262378s FAIL (2025-09-23 21:26:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.67265541s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.755133736s FAIL (2025-09-23 21:25:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.755161635s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.54024476s FAIL (2025-09-23 21:24:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.540278025s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.680920065s FAIL (2025-09-23 21:23:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.680949919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.592880009s FAIL (2025-09-23 21:22:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.592908643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.678311184s FAIL (2025-09-23 21:21:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.678346722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.488905475s FAIL (2025-09-23 21:21:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.488957976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.599357012s FAIL (2025-09-23 21:20:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.599384006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.571239317s FAIL (2025-09-23 21:19:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.571271777s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.643883055s FAIL (2025-09-23 21:18:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.643910198s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.589272289s FAIL (2025-09-23 21:17:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.589357562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.421376835s FAIL (2025-09-23 21:17:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.421470249s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.637008171s FAIL (2025-09-23 21:16:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.637058142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.633742002s FAIL (2025-09-23 21:15:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.633778262s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.587493132s FAIL (2025-09-23 21:15:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.587523035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.59829357s FAIL (2025-09-23 21:14:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.598323076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.66130483s FAIL (2025-09-23 21:13:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.661376768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.481121093s FAIL (2025-09-23 21:12:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.481171408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.565835796s FAIL (2025-09-23 21:12:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 16.565889646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.107830066s FAIL (2025-09-23 21:11:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== U4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.620851426 (7h41m2.424316004s) [seg_pos={0 0}, upload=6696]: Done begin upload segment. Sep 23 21:11:14.620868431 (7h41m2.42433303s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.621868107 (7h41m2.425332698s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622016702 (7h41m2.425481284s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622404300 (7h41m2.4258689s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.622724836 (7h41m2.42618943s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623032294 (7h41m2.426496885s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623380095 (7h41m2.42684469s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.623737696 (7h41m2.427202287s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624189892 (7h41m2.427654492s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624583949 (7h41m2.428048542s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.624924475 (7h41m2.428389069s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.625246922 (7h41m2.428711518s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627166428 (7h41m2.430631022s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.627538796 (7h41m2.431003378s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:14.628090838 (7h41m2.431555439s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Uploading piece... Sep 23 21:11:15.288555663 (7h41m3.092020254s) [upload=6696]: Next returned nil segment Sep 23 21:11:15.288574221 (7h41m3.092038815s) [upload=6696]: Waiting for error group managing segments... Sep 23 21:11:15.421108705 (7h41m3.224573286s) [noise=true , address=145.79.227.17:4970 , piece_id=5S4DRVTIXWVGEEFUI6YLGEALUJYZYJNGSAJICUN2KQKP7VXO4PSQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422545800 (7h41m3.226010385s) [noise=true , address=145.79.227.3:4151 , piece_id=CLQHIV3BZH62CBYZWS4YW3TPPOUNPFRQ33I575MK4NQ6N7F4B5UA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.422917871 (7h41m3.226382453s) [noise=true , address=145.79.227.16:4909 , piece_id=HQTAS2KSJ4KSMAVQEUTEBAVP3JANOLTDRRJD3OB2SKFPVY5FRRVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.430292178 (7h41m3.233756762s) [noise=true , address=145.79.227.7:4295 , piece_id=FAAAOUEE4MBFAFO5V4PQAIR5EDKZQP6FGVWTSZKAMHFAABHCZ35Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.442794615 (7h41m3.246259201s) [noise=true , address=145.79.227.18:4006 , piece_id=R5TSYHLBNTAZQ2C4CQFFNRTLTX3W7R63NI5JOOZ5WTKDH56Y3IKQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444271469 (7h41m3.24773605s) [noise=true , address=145.79.227.2:4114 , piece_id=NY5HJK6YOI2DLF6HL77VEI5QFFGGKANKGJK5E5C3QQGW3FQBQTLA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.444863505 (7h41m3.2483281s) [noise=true , address=145.79.227.11:4732 , piece_id=DY6WKQYZ6TLLCTO5PDRTS5LAJEJVZTG4OIL5M5GIQB2JFPE23UTA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.447669812 (7h41m3.251134405s) [noise=true , address=145.79.227.2:4115 , piece_id=UHHURCTLMSIV63KHMLYKLYJ43L2ACWJPWQL6UKG7IT6TR4XE5IOQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:15.454590085 (7h41m3.258054667s) [noise=true , address=145.79.227.17:4972 , piece_id=PZOEN6W7YDRVNT52SCCSRLQMGZXM2337NU4D5DGZ43Q7TTDKI7VQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: <nil> Sep 23 21:11:44.136734291 (7h41m31.940198892s) [noise=true , address=45.91.116.13:4154 , piece_id=PJY75N4XXNQ5UWTKNUEFNW5PKT23KG5FYRSMFZIIJ6IELQX4JG2A, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57702->45.91.116.13:4154: use of closed network connection Sep 23 21:11:44.136764699 (7h41m31.940229542s) [noise=true , address=45.91.116.11:4119 , piece_id=7NXITJOXFNOO2Z3LLZR7DE6KIXOTO5EQILZOAK43WIZKZDNTB6GQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57160->45.91.116.11:4119: use of closed network connection Sep 23 21:11:44.136877628 (7h41m31.940342219s) [noise=true , address=45.91.116.6:4974 , piece_id=XGTZXFZCLYFAXV7CAMLYS24DWKKFVGFSCM23YGRNO4AUMUBXYCHQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33628->45.91.116.6:4974: use of closed network connection Sep 23 21:11:44.136937741 (7h41m31.940402334s) [noise=true , address=77.90.31.18:4806 , piece_id=WOLJYSYGRSZIU4QJR7E72VBLSLFXEH23YTLXOBO5D3DAXXFVA2PA, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32906->77.90.31.18:4806: use of closed network connection Sep 23 21:11:44.137036518 (7h41m31.940501109s) [noise=true , address=45.91.116.5:4299 , piece_id=CQSQQ2MEZWVRVYTGAIA77KTYP7MQWYJTZJJKSJZSSXZ52N4KRJVQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57668->45.91.116.5:4299: use of closed network connection Sep 23 21:11:44.137079619 (7h41m31.940544211s) [noise=true , address=45.91.116.10:4081 , piece_id=IF64EUH2ML3LUHA3UYNCZJM7FPRCZ3WRL4UNREVXI7O3TVRJMT3Q, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34466->45.91.116.10:4081: use of closed network connection Sep 23 21:11:44.137233059 (7h41m31.940697654s) [noise=true , address=45.91.116.8:4048 , piece_id=MBS2RTF37W24SLRVYASLU7UHA6ZBKAMBBF5RK635LL3SJOBQC7OQ, seg_pos={0 0}, upload=6696]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43322->45.91.116.8:4048: use of closed network connection Sep 23 21:11:44.137264479 (7h41m31.940729072s) [seg_pos={0 0}, upload=6696]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 23 21:11:44.243070747 (7h41m32.046535333s) [upload=6696]: Done uploading segments... Sep 23 21:11:44.243102526 (7h41m32.046567121s) [upload=6696]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.107857295s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.796665483s FAIL (2025-09-23 19:32:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.796687878s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.877096416s FAIL (2025-09-23 19:32:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.877119682s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 2.735562506s FAIL (2025-09-23 19:31:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.735617212s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 2.772134532s FAIL (2025-09-23 19:31:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.772160303s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 2.654279761s FAIL (2025-09-23 19:30:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.654392195s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.52132795s FAIL (2025-09-23 19:30:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.521371049s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.546687559s FAIL (2025-09-23 19:30:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.54673243s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 2.729703313s FAIL (2025-09-23 19:29:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.729806832s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.387064418s FAIL (2025-09-23 19:29:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.387095286s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.600550402s FAIL (2025-09-23 19:29:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.600654728s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 2.792574575s FAIL (2025-09-23 19:29:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.792608541s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.612322474s FAIL (2025-09-23 19:28:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.612361575s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 2.787739703s FAIL (2025-09-23 19:28:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.787763927s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.587434161s FAIL (2025-09-23 19:27:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.58746451s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XXXX...................................................] duration: 2.660790115s FAIL (2025-09-23 19:27:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.660819044s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.526244052s FAIL (2025-09-23 19:26:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 23 18:48:27.071761249 (5h18m14.87522583s) [upload=4632]: Starting upload Sep 23 18:48:27.071844498 (5h18m14.875309093s) [upload=4632]: Done starting upload Sep 23 18:48:27.071922976 (5h18m14.875387554s) [upload=4632]: Uploading segments... Sep 23 18:48:27.072100574 (5h18m14.875565167s) [upload=4632]: Got next segment. Inline: false Sep 23 18:48:27.072169755 (5h18m14.875634335s) [upload=4632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 18:48:27.072175756 (5h18m14.875640347s) [upload=4632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 18:48:27.252485055 (5h18m15.055949656s) [seg_pos={0 0}, upload=4632]: Begin upload segment... Sep 23 18:48:27.252544796 (5h18m15.056009395s) [seg_pos={0 0}, upload=4632]: Done begin upload segment. Sep 23 18:48:27.252574818 (5h18m15.056039422s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.259786732 (5h18m15.063251334s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.261424618 (5h18m15.064889219s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.263205946 (5h18m15.066670542s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.264985460 (5h18m15.068450057s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.266786187 (5h18m15.070250891s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.268394959 (5h18m15.071859716s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.269229524 (5h18m15.072694123s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.270092520 (5h18m15.07355712s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.271066258 (5h18m15.074530853s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.272668000 (5h18m15.076132603s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.273266904 (5h18m15.076731504s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275204173 (5h18m15.078668775s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.275908609 (5h18m15.079373205s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277231628 (5h18m15.080696205s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.277884907 (5h18m15.081349502s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Uploading piece... Sep 23 18:48:27.881885858 (5h18m15.685350434s) [upload=4632]: Next returned nil segment Sep 23 18:48:27.881905136 (5h18m15.685369726s) [upload=4632]: Waiting for error group managing segments... Sep 23 18:48:28.016523029 (5h18m15.819987622s) [noise=true , address=45.91.116.14:4189 , piece_id=7Z7GT4GYOJKD2NVOD5EKBTRYOLU56EVJ7OKZVMM6K6ZMEP367IPQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.017292495 (5h18m15.820757079s) [noise=true , address=45.91.116.6:4975 , piece_id=6OVX6X3KIPEHF5RGOGCJKB2H4AM4LJRROLPC6AQN5FZ5JB34YAQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.018428684 (5h18m15.821893282s) [noise=true , address=45.91.116.5:4298 , piece_id=KXNGXZSYDYEXSZ5T5KJLKC6PIQYWJXG7RGGAR6J3NXLICTNXHXVA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019532942 (5h18m15.822997538s) [noise=true , address=45.91.116.4:4939 , piece_id=NQOZP6NJ4AHBAZAUXRBKJRULEFIIKOVAQC4CJF4RCOEBZWWRU2CA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.019840213 (5h18m15.823304813s) [noise=true , address=45.91.116.12:4225 , piece_id=YWPRV3UHCYZ7ZK6FB2FAOE6BU3DE4QHF5DJADO3LVTASCXAL425Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.029655046 (5h18m15.833119627s) [noise=true , address=45.91.116.8:4047 , piece_id=BCK33CBO6B3GWUNEDSYDJLZCF5CLRAWFH5JZ6UOEHV3GOSMEPLQA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.031565646 (5h18m15.835030238s) [noise=true , address=77.90.31.16:4734 , piece_id=63QQF7XI6T5KX7XB5MSEVS4ZY4ZZ5WDKWLGMJL3YUO4URB4N3MKQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.038750186 (5h18m15.842214791s) [noise=true , address=77.90.31.10:4520 , piece_id=WNUH6FGATWV2WPRW7DVLF3UPYP5PGGBLEPYITMEP45PYQPXGPBOQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:28.052415828 (5h18m15.855880418s) [noise=true , address=77.90.31.17:4771 , piece_id=MWME5UAEW2KC2PYZR54SNHO6EFWSB35SXAZRSTSANHRUKOFQ3F2Q, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:40.435844938 (5h18m28.23930953s) [noise=true , address=145.79.227.15:4875 , piece_id=6KZBL6OHCEMWRCXUS7J2HEOACCINFH26AWOIABOIOABFLQL2UTEQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:53.946852052 (5h18m41.750316644s) [noise=true , address=145.79.227.11:4732 , piece_id=4Q462223V752OAQY5H5O4K4TBYACIERWKS3YPOV6GMY2JDR42ORQ, seg_pos={0 0}, upload=4632]: Done uploading piece. err: <nil> Sep 23 18:48:56.762365912 (5h18m44.565830506s) [noise=true , address=145.79.227.13:4804 , piece_id=2GOLBVFQ7BMGPRS52CVO6R346FPSZCFV7K5WDWKARRPC4AKOMZXA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762470617 (5h18m44.565935202s) [noise=true , address=145.79.227.18:4007 , piece_id=SVNRCB6DJHAWINQHQWNIXTZP4YBHEZT4OTQ4SU47NBTMP5HFZHNA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:53046->145.79.227.18:4007: use of closed network connection Sep 23 18:48:56.762497707 (5h18m44.565962299s) [noise=true , address=145.79.227.7:4295 , piece_id=UUH3PVGX76DSR2WPLF5IASLQPX745DRG7XTTSEOJMBVF25CEB5OA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56488->145.79.227.7:4295: use of closed network connection Sep 23 18:48:56.762569129 (5h18m44.566033721s) [noise=true , address=145.79.227.12:4767 , piece_id=5UIPNF6WPPEYSBFI2NLE4MWC6W2F6IP3MYUQSET3UQFK25NG4K6A, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762634698 (5h18m44.566099291s) [noise=true , address=145.79.227.16:4910 , piece_id=KE4O64FTIHRT53TOUL5A2OQ7JFQZGSM6GM7K5T65NW24N4PEP7EA, seg_pos={0 0}, upload=4632]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: expected piece hash; context deadline exceeded; EOF Sep 23 18:48:56.762660559 (5h18m44.566125152s) [seg_pos={0 0}, upload=4632]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 23 18:48:56.880060680 (5h18m44.683525262s) [upload=4632]: Done uploading segments... Sep 23 18:48:56.880087013 (5h18m44.683551605s) [upload=4632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.526273007s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XXXX..................................................] duration: 2.804211446s FAIL (2025-09-23 19:26:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.80423924s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)