Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..............XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..] duration: 2.718226219s FAIL (2025-09-22 02:43:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:41:36.601496414 (564h33m57.093928844s) [upload=496660]: Starting upload Sep 22 02:41:36.601577065 (564h33m57.094009506s) [upload=496660]: Done starting upload Sep 22 02:41:36.601709764 (564h33m57.094142192s) [upload=496660]: Uploading segments... Sep 22 02:41:36.601998385 (564h33m57.09443083s) [upload=496660]: Got next segment. Inline: false Sep 22 02:41:36.602062308 (564h33m57.094494736s) [upload=496660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:41:36.602067200 (564h33m57.094499643s) [upload=496660]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:41:36.726653879 (564h33m57.219086324s) [seg_pos={0 0}, upload=496660]: Begin upload segment... Sep 22 02:41:36.726715101 (564h33m57.219147533s) [seg_pos={0 0}, upload=496660]: Done begin upload segment. Sep 22 02:41:36.726763835 (564h33m57.219196283s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727199512 (564h33m57.219631959s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727453253 (564h33m57.219885698s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727696529 (564h33m57.220128978s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728023384 (564h33m57.220455831s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728315482 (564h33m57.220747928s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728545463 (564h33m57.220977904s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728761945 (564h33m57.221194388s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.729982144 (564h33m57.222414589s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.730761341 (564h33m57.223193796s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731236380 (564h33m57.223668831s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731732211 (564h33m57.224164661s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734110494 (564h33m57.226542944s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734499844 (564h33m57.226932287s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734841727 (564h33m57.22727417s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.735192143 (564h33m57.227624595s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:37.345408248 (564h33m57.83784069s) [upload=496660]: Next returned nil segment Sep 22 02:41:37.345426076 (564h33m57.837858508s) [upload=496660]: Waiting for error group managing segments... Sep 22 02:41:37.479696888 (564h33m57.972129325s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480877864 (564h33m57.973310312s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480955327 (564h33m57.97338777s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.503208075 (564h33m57.995640515s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.514556726 (564h33m58.006989154s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.515929308 (564h33m58.008361737s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.521418086 (564h33m58.013850518s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.548174593 (564h33m58.040607028s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.552857975 (564h33m58.045290421s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.608476999 (564h33m58.100909427s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:40.424312625 (564h34m0.916745062s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302708895 (564h34m26.795141331s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302967978 (564h34m26.795400413s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303041063 (564h34m26.795473499s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303092578 (564h34m26.795525022s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303121911 (564h34m26.79555434s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303149863 (564h34m26.795582296s) [seg_pos={0 0}, upload=496660]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:42:06.413791612 (564h34m26.906224057s) [upload=496660]: Done uploading segments... Sep 22 02:42:06.413827106 (564h34m26.906259538s) [upload=496660]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.71826999s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............XXXX.X....X..................................] duration: 2.694578886s FAIL (2025-09-22 02:43:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.694616486s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......................X.X.......X..X......X.X.....X.X...XX.] duration: 2.603076478s FAIL (2025-09-22 02:42:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.603145524s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........................X.X.......X..X......X.X.....X.X...XX] duration: 30.000902841s FAIL (2025-09-22 02:42:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000918753s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........................X.X.......X..X......X.X.....X.X...X] duration: 30.001255166s FAIL (2025-09-22 02:42:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001283583s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [...............XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X.] duration: 2.346102015s FAIL (2025-09-22 02:42:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:41:36.601496414 (564h33m57.093928844s) [upload=496660]: Starting upload Sep 22 02:41:36.601577065 (564h33m57.094009506s) [upload=496660]: Done starting upload Sep 22 02:41:36.601709764 (564h33m57.094142192s) [upload=496660]: Uploading segments... Sep 22 02:41:36.601998385 (564h33m57.09443083s) [upload=496660]: Got next segment. Inline: false Sep 22 02:41:36.602062308 (564h33m57.094494736s) [upload=496660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:41:36.602067200 (564h33m57.094499643s) [upload=496660]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:41:36.726653879 (564h33m57.219086324s) [seg_pos={0 0}, upload=496660]: Begin upload segment... Sep 22 02:41:36.726715101 (564h33m57.219147533s) [seg_pos={0 0}, upload=496660]: Done begin upload segment. Sep 22 02:41:36.726763835 (564h33m57.219196283s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727199512 (564h33m57.219631959s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727453253 (564h33m57.219885698s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727696529 (564h33m57.220128978s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728023384 (564h33m57.220455831s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728315482 (564h33m57.220747928s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728545463 (564h33m57.220977904s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728761945 (564h33m57.221194388s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.729982144 (564h33m57.222414589s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.730761341 (564h33m57.223193796s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731236380 (564h33m57.223668831s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731732211 (564h33m57.224164661s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734110494 (564h33m57.226542944s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734499844 (564h33m57.226932287s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734841727 (564h33m57.22727417s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.735192143 (564h33m57.227624595s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:37.345408248 (564h33m57.83784069s) [upload=496660]: Next returned nil segment Sep 22 02:41:37.345426076 (564h33m57.837858508s) [upload=496660]: Waiting for error group managing segments... Sep 22 02:41:37.479696888 (564h33m57.972129325s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480877864 (564h33m57.973310312s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480955327 (564h33m57.97338777s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.503208075 (564h33m57.995640515s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.514556726 (564h33m58.006989154s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.515929308 (564h33m58.008361737s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.521418086 (564h33m58.013850518s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.548174593 (564h33m58.040607028s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.552857975 (564h33m58.045290421s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.608476999 (564h33m58.100909427s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:40.424312625 (564h34m0.916745062s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302708895 (564h34m26.795141331s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302967978 (564h34m26.795400413s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303041063 (564h34m26.795473499s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303092578 (564h34m26.795525022s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303121911 (564h34m26.79555434s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303149863 (564h34m26.795582296s) [seg_pos={0 0}, upload=496660]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:42:06.413791612 (564h34m26.906224057s) [upload=496660]: Done uploading segments... Sep 22 02:42:06.413827106 (564h34m26.906259538s) [upload=496660]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 2.346132467s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X] duration: 30.113851546s FAIL (2025-09-22 02:42:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:41:36.601496414 (564h33m57.093928844s) [upload=496660]: Starting upload Sep 22 02:41:36.601577065 (564h33m57.094009506s) [upload=496660]: Done starting upload Sep 22 02:41:36.601709764 (564h33m57.094142192s) [upload=496660]: Uploading segments... Sep 22 02:41:36.601998385 (564h33m57.09443083s) [upload=496660]: Got next segment. Inline: false Sep 22 02:41:36.602062308 (564h33m57.094494736s) [upload=496660]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:41:36.602067200 (564h33m57.094499643s) [upload=496660]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:41:36.726653879 (564h33m57.219086324s) [seg_pos={0 0}, upload=496660]: Begin upload segment... Sep 22 02:41:36.726715101 (564h33m57.219147533s) [seg_pos={0 0}, upload=496660]: Done begin upload segment. Sep 22 02:41:36.726763835 (564h33m57.219196283s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727199512 (564h33m57.219631959s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727453253 (564h33m57.219885698s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.727696529 (564h33m57.220128978s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728023384 (564h33m57.220455831s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728315482 (564h33m57.220747928s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728545463 (564h33m57.220977904s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.728761945 (564h33m57.221194388s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.729982144 (564h33m57.222414589s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.730761341 (564h33m57.223193796s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731236380 (564h33m57.223668831s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.731732211 (564h33m57.224164661s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734110494 (564h33m57.226542944s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734499844 (564h33m57.226932287s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.734841727 (564h33m57.22727417s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:36.735192143 (564h33m57.227624595s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Uploading piece... Sep 22 02:41:37.345408248 (564h33m57.83784069s) [upload=496660]: Next returned nil segment Sep 22 02:41:37.345426076 (564h33m57.837858508s) [upload=496660]: Waiting for error group managing segments... Sep 22 02:41:37.479696888 (564h33m57.972129325s) [noise=true , address=45.91.116.7:4010 , piece_id=OOLZFYLFJ2DAKJ2ISGRT5D5UUH5OQ6OB6WJGCMGRSXECON5RDT6Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480877864 (564h33m57.973310312s) [noise=true , address=45.91.116.10:4081 , piece_id=LOQFBHR7W63AQXMX44SH72NP5FE5PPL27V3FQKWH5SKNGRDNNYKA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.480955327 (564h33m57.97338777s) [noise=true , address=77.90.31.21:4877 , piece_id=Q5FSM73UA5C37T2W7DEU4M4KJOYJJ2ZK5VVGZDLF5SWYXXISU2KQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.503208075 (564h33m57.995640515s) [noise=true , address=45.91.116.4:4937 , piece_id=ENLSGU2TLR6RB4BSHT6E3W7FUO7SWCYNO73TT35YPNEXSWHDWMWA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.514556726 (564h33m58.006989154s) [noise=true , address=77.90.31.13:4627 , piece_id=E45BALYV6XNEFYXMRH7C3O7AZVTPZYN44ABST4NRLWHOPECDJ4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.515929308 (564h33m58.008361737s) [noise=true , address=45.91.116.8:4048 , piece_id=CYEZY275B4V3XN7YEMFKXWV54LUR6WCNHIW5OMREYYPQKQF6OHZQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.521418086 (564h33m58.013850518s) [noise=true , address=45.91.116.9:4261 , piece_id=EDKNNVNXNSJNAVYNKSYBGWHCIRHQBP7QE7GWFUTR3AKA4OANIOOA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.548174593 (564h33m58.040607028s) [noise=true , address=45.91.116.13:4154 , piece_id=AHTFMCBYAMPRPJOCTO3RR6IUWJY543IB6IJDIIAZVVLDJJWBTJYA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.552857975 (564h33m58.045290421s) [noise=true , address=45.91.116.6:4973 , piece_id=UYIYAMGJJDBDKFKVKJC4J7W5GOLOZA3OMLBRIDKQXLF5AQPWIXQQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:37.608476999 (564h33m58.100909427s) [noise=true , address=45.91.116.13:4153 , piece_id=D6QFCVINYCPC4GKR3JV5R4OEQPSXJ77JR6T42JM7OFAHVS7BPWSA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: <nil> Sep 22 02:41:40.424312625 (564h34m0.916745062s) [noise=true , address=145.79.227.11:4729 , piece_id=B3UW7KPYXVSULRB7JFTDECOR3IKR2KSPZAG7QU2OYHJQWQB6REHQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302708895 (564h34m26.795141331s) [noise=true , address=145.79.227.6:4257 , piece_id=REK2IK54CRE7DYBNWBXQ3HPXSW4GPGMYZZCGYELBEQ66A6JBNMWQ, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.302967978 (564h34m26.795400413s) [noise=true , address=145.79.227.16:4912 , piece_id=MRMZSDZ5ZIFVLMMJP3OFBNQTJZJGHFZGKA4SLBLOOL7V74QS4BFA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303041063 (564h34m26.795473499s) [noise=true , address=145.79.227.3:4151 , piece_id=D6QFMIYPX6EOAMEC4NYTWYO5QDVVMNR5G74YOQBXDFIFEVVARNEA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303092578 (564h34m26.795525022s) [noise=true , address=145.79.227.4:4187 , piece_id=BIH2GDGVYLPPJR2R7XT4MWARW4MQBAEFBAO6AW2MY3YE67WTG4KA, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303121911 (564h34m26.79555434s) [noise=true , address=145.79.227.5:4222 , piece_id=QXLLSDGMJBLSZBPOH6USCJSRI7OLQXLGX7LGCSVWOWWFO5C2DR4Q, seg_pos={0 0}, upload=496660]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:42:06.303149863 (564h34m26.795582296s) [seg_pos={0 0}, upload=496660]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:42:06.413791612 (564h34m26.906224057s) [upload=496660]: Done uploading segments... Sep 22 02:42:06.413827106 (564h34m26.906259538s) [upload=496660]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.113876976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [................XXXX.X....X.................................] duration: 2.618468277s FAIL (2025-09-22 02:41:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.618494396s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................XXXX.X....X................................] duration: 2.460637964s FAIL (2025-09-22 02:41:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.460659102s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................X.X.......X..X......X.X.....X.X...] duration: 2.940507371s FAIL (2025-09-22 02:41:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.940544401s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..] duration: 2.487734245s FAIL (2025-09-22 02:40:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:39:40.153785546 (564h32m0.64621799s) [upload=496632]: Starting upload Sep 22 02:39:40.153863837 (564h32m0.646296271s) [upload=496632]: Done starting upload Sep 22 02:39:40.154154068 (564h32m0.646586502s) [upload=496632]: Uploading segments... Sep 22 02:39:40.154372012 (564h32m0.646804445s) [upload=496632]: Got next segment. Inline: false Sep 22 02:39:40.154405706 (564h32m0.646838139s) [upload=496632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:39:40.154411002 (564h32m0.646843444s) [upload=496632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:39:40.265520364 (564h32m0.757952806s) [seg_pos={0 0}, upload=496632]: Begin upload segment... Sep 22 02:39:40.265571157 (564h32m0.758003599s) [seg_pos={0 0}, upload=496632]: Done begin upload segment. Sep 22 02:39:40.265600035 (564h32m0.75803248s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265617565 (564h32m0.758050011s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265998982 (564h32m0.758431426s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266155443 (564h32m0.758587889s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266357323 (564h32m0.758789771s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266507631 (564h32m0.758940073s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266634504 (564h32m0.75906695s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266907978 (564h32m0.759340421s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.267232369 (564h32m0.759664814s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.268553429 (564h32m0.760985878s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.269666500 (564h32m0.762098948s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270309597 (564h32m0.762742044s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270655878 (564h32m0.76308832s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271038182 (564h32m0.76347063s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271412526 (564h32m0.763844968s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271764408 (564h32m0.764196841s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.881360839 (564h32m1.373793274s) [upload=496632]: Next returned nil segment Sep 22 02:39:40.881378017 (564h32m1.373810459s) [upload=496632]: Waiting for error group managing segments... Sep 22 02:39:41.018936593 (564h32m1.511369039s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020029680 (564h32m1.512462126s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020818101 (564h32m1.513250537s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023506081 (564h32m1.51593851s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023768394 (564h32m1.516200829s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023864972 (564h32m1.5162974s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.025833617 (564h32m1.518266052s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.026470835 (564h32m1.518903264s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.028559621 (564h32m1.520992056s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.029649413 (564h32m1.522081847s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.030949688 (564h32m1.523382123s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.044318879 (564h32m1.536751314s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:40:09.866638070 (564h32m30.359070506s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866718163 (564h32m30.359150594s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866768146 (564h32m30.359200579s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866820134 (564h32m30.359252566s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866853798 (564h32m30.359286248s) [seg_pos={0 0}, upload=496632]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:40:09.981166007 (564h32m30.473598442s) [upload=496632]: Done uploading segments... Sep 22 02:40:09.981223626 (564h32m30.473656069s) [upload=496632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.487769003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................XXXX.X....X...............................] duration: 2.426535744s FAIL (2025-09-22 02:40:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.426564944s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X.] duration: 2.576406018s FAIL (2025-09-22 02:40:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:39:40.153785546 (564h32m0.64621799s) [upload=496632]: Starting upload Sep 22 02:39:40.153863837 (564h32m0.646296271s) [upload=496632]: Done starting upload Sep 22 02:39:40.154154068 (564h32m0.646586502s) [upload=496632]: Uploading segments... Sep 22 02:39:40.154372012 (564h32m0.646804445s) [upload=496632]: Got next segment. Inline: false Sep 22 02:39:40.154405706 (564h32m0.646838139s) [upload=496632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:39:40.154411002 (564h32m0.646843444s) [upload=496632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:39:40.265520364 (564h32m0.757952806s) [seg_pos={0 0}, upload=496632]: Begin upload segment... Sep 22 02:39:40.265571157 (564h32m0.758003599s) [seg_pos={0 0}, upload=496632]: Done begin upload segment. Sep 22 02:39:40.265600035 (564h32m0.75803248s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265617565 (564h32m0.758050011s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265998982 (564h32m0.758431426s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266155443 (564h32m0.758587889s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266357323 (564h32m0.758789771s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266507631 (564h32m0.758940073s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266634504 (564h32m0.75906695s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266907978 (564h32m0.759340421s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.267232369 (564h32m0.759664814s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.268553429 (564h32m0.760985878s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.269666500 (564h32m0.762098948s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270309597 (564h32m0.762742044s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270655878 (564h32m0.76308832s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271038182 (564h32m0.76347063s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271412526 (564h32m0.763844968s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271764408 (564h32m0.764196841s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.881360839 (564h32m1.373793274s) [upload=496632]: Next returned nil segment Sep 22 02:39:40.881378017 (564h32m1.373810459s) [upload=496632]: Waiting for error group managing segments... Sep 22 02:39:41.018936593 (564h32m1.511369039s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020029680 (564h32m1.512462126s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020818101 (564h32m1.513250537s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023506081 (564h32m1.51593851s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023768394 (564h32m1.516200829s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023864972 (564h32m1.5162974s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.025833617 (564h32m1.518266052s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.026470835 (564h32m1.518903264s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.028559621 (564h32m1.520992056s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.029649413 (564h32m1.522081847s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.030949688 (564h32m1.523382123s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.044318879 (564h32m1.536751314s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:40:09.866638070 (564h32m30.359070506s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866718163 (564h32m30.359150594s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866768146 (564h32m30.359200579s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866820134 (564h32m30.359252566s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866853798 (564h32m30.359286248s) [seg_pos={0 0}, upload=496632]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:40:09.981166007 (564h32m30.473598442s) [upload=496632]: Done uploading segments... Sep 22 02:40:09.981223626 (564h32m30.473656069s) [upload=496632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576440685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X] duration: 30.11709506s FAIL (2025-09-22 02:40:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:39:40.153785546 (564h32m0.64621799s) [upload=496632]: Starting upload Sep 22 02:39:40.153863837 (564h32m0.646296271s) [upload=496632]: Done starting upload Sep 22 02:39:40.154154068 (564h32m0.646586502s) [upload=496632]: Uploading segments... Sep 22 02:39:40.154372012 (564h32m0.646804445s) [upload=496632]: Got next segment. Inline: false Sep 22 02:39:40.154405706 (564h32m0.646838139s) [upload=496632]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:39:40.154411002 (564h32m0.646843444s) [upload=496632]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:39:40.265520364 (564h32m0.757952806s) [seg_pos={0 0}, upload=496632]: Begin upload segment... Sep 22 02:39:40.265571157 (564h32m0.758003599s) [seg_pos={0 0}, upload=496632]: Done begin upload segment. Sep 22 02:39:40.265600035 (564h32m0.75803248s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265617565 (564h32m0.758050011s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.265998982 (564h32m0.758431426s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266155443 (564h32m0.758587889s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266357323 (564h32m0.758789771s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266507631 (564h32m0.758940073s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266634504 (564h32m0.75906695s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.266907978 (564h32m0.759340421s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.267232369 (564h32m0.759664814s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.268553429 (564h32m0.760985878s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.269666500 (564h32m0.762098948s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270309597 (564h32m0.762742044s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.270655878 (564h32m0.76308832s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271038182 (564h32m0.76347063s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271412526 (564h32m0.763844968s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.271764408 (564h32m0.764196841s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Uploading piece... Sep 22 02:39:40.881360839 (564h32m1.373793274s) [upload=496632]: Next returned nil segment Sep 22 02:39:40.881378017 (564h32m1.373810459s) [upload=496632]: Waiting for error group managing segments... Sep 22 02:39:41.018936593 (564h32m1.511369039s) [noise=true , address=45.91.116.10:4081 , piece_id=SD63CBVKVYG2ZPMVSV7AHLADD4N6NE3CIVQEVHDTS4BGJTZMGJ4A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020029680 (564h32m1.512462126s) [noise=true , address=45.91.116.9:4263 , piece_id=RJSHX3UXC5FNQPT7OYNUMVGMZMNKZ4SUEXW75IBAEF7VQXCDLQPA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.020818101 (564h32m1.513250537s) [noise=true , address=77.90.31.12:4589 , piece_id=VYZ7P6FEH4ERFBSRLPLFEFUD4U4AVHVCTHD3S6MAT656NR7Q267A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023506081 (564h32m1.51593851s) [noise=true , address=45.91.116.5:4297 , piece_id=ZZUX2CDLQ4TTK5XXHHR76AYPV7ITBSBPDRIWPXOLTQOS3OIJY4JA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023768394 (564h32m1.516200829s) [noise=true , address=77.90.31.13:4628 , piece_id=3YT4XNVORFFFJLWKXQXRLG63HZ6FG4WPJH7SRSTSLYZL6ILZRTMQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.023864972 (564h32m1.5162974s) [noise=true , address=77.90.31.19:4842 , piece_id=UMWKKDWZHBRK6HDKED6DRUGP234THRK7JTH6WI6JB3AII4SBURXQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.025833617 (564h32m1.518266052s) [noise=true , address=45.91.116.12:4225 , piece_id=RA2L5PQSPIKSHJ6TI7J3B6MRG7JVYXJHVA4TETQK6MC4ZUXCPVOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.026470835 (564h32m1.518903264s) [noise=true , address=45.91.116.4:4938 , piece_id=Q53CD3ECCFYVLXLQXBIBUEQ5C3NYTO2BXBP54TGH2O652BRA5DAQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.028559621 (564h32m1.520992056s) [noise=true , address=77.90.31.14:4663 , piece_id=RAZV57JBDLKUSIEE4YTOUP3LPGCPTP7HPEOQK44JOOUFYTVMGMHQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.029649413 (564h32m1.522081847s) [noise=true , address=45.91.116.13:4153 , piece_id=DHBZCW5NYKIXBWBQLG2GB6UKLAYFN3OSIVDIV7GUKDLVD6FAD2UQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.030949688 (564h32m1.523382123s) [noise=true , address=45.91.116.8:4047 , piece_id=EGGTJWBVQV6KWOXHL52IOEWVOVOV2XIMKOP4E37FCJ4B3EIJLZDQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:39:41.044318879 (564h32m1.536751314s) [noise=true , address=77.90.31.11:4554 , piece_id=ZUFFTZEQCGHJ7OAKG55PRQM4EMGZFHOH7EELFSLJVKB6EZJEKXOQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: <nil> Sep 22 02:40:09.866638070 (564h32m30.359070506s) [noise=true , address=145.79.227.12:4765 , piece_id=D5GHUBHLTUZZVBXKS72QVZY4PD632OCEEAQ3SW6IJFLMTKTC46KA, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866718163 (564h32m30.359150594s) [noise=true , address=145.79.227.13:4801 , piece_id=5CRAA6BNBRO535R5NTKKRFCZPO7VH6I5NFSLCWM6WWQHNXOLYR3A, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866768146 (564h32m30.359200579s) [noise=true , address=145.79.227.11:4730 , piece_id=WUYHINB777OE5WYDEBUWDVXALCL7D64VSN7B7BZ6AGZUMGIQQLFQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866820134 (564h32m30.359252566s) [noise=true , address=145.79.227.2:4115 , piece_id=KIDIMUEK75VNCFKBIMRYIAYU6P2XW77A77W5OPKD5464FWOGXPTQ, seg_pos={0 0}, upload=496632]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:40:09.866853798 (564h32m30.359286248s) [seg_pos={0 0}, upload=496632]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:40:09.981166007 (564h32m30.473598442s) [upload=496632]: Done uploading segments... Sep 22 02:40:09.981223626 (564h32m30.473656069s) [upload=496632]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.117140282s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........................X.X.......X..X......X.X.....X.X..] duration: 2.975567731s FAIL (2025-09-22 02:40:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.975595102s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................XXXX.X....X..............................] duration: 2.87218514s FAIL (2025-09-22 02:39:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.872214964s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................X.X.......X..X......X.X.....X.X.] duration: 2.62144292s FAIL (2025-09-22 02:39:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.621485858s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............................X.X.......X..X......X.X.....X.X] duration: 30.001410746s FAIL (2025-09-22 02:39:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001438299s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [....................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........] duration: 2.375725639s FAIL (2025-09-22 02:39:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.375759229s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................XXXX.X....X.............................] duration: 2.47923686s FAIL (2025-09-22 02:39:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.479266178s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............................X.X.......X..X......X.X.....X.] duration: 2.928036139s FAIL (2025-09-22 02:38:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.928060705s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.....................XXXXXXX..X.X...XXXXXXX.X.X.X.XX........] duration: 2.51053366s FAIL (2025-09-22 02:38:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.510582244s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................XXXX.X....X............................] duration: 2.564378032s FAIL (2025-09-22 02:38:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.564473847s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............................X.X.......X..X......X.X.....X] duration: 30.000719355s FAIL (2025-09-22 02:37:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000746789s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [......................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.......] duration: 2.418736563s FAIL (2025-09-22 02:37:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.41877909s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................X.X.......X..X......X.X.....] duration: 2.703807755s FAIL (2025-09-22 02:37:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.703854268s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................XXXX.X....X...........................] duration: 2.776567688s FAIL (2025-09-22 02:36:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.776745s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.......................XXXXXXX..X.X...XXXXXXX.X.X.X.XX......] duration: 2.656734398s FAIL (2025-09-22 02:36:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.65684115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................XXXX.X....X..........................] duration: 2.488807951s FAIL (2025-09-22 02:36:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.488843109s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................X.X.......X..X......X.X....] duration: 2.720797945s FAIL (2025-09-22 02:35:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.720824566s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................XXXX.X....X.........................] duration: 2.663193455s FAIL (2025-09-22 02:35:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.663226898s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [........................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.....] duration: 2.552958574s FAIL (2025-09-22 02:35:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.553060519s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................X.X.......X..X......X.X...] duration: 2.904334266s FAIL (2025-09-22 02:35:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.904388677s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................XXXX.X....X........................] duration: 2.627131268s FAIL (2025-09-22 02:35:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.62716441s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................XXXX.X....X.......................] duration: 2.65553983s FAIL (2025-09-22 02:34:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.655577732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.........................XXXXXXX..X.X...XXXXXXX.X.X.X.XX....] duration: 2.407903677s FAIL (2025-09-22 02:34:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.407934783s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................X.X.......X..X......X.X..] duration: 2.990607512s FAIL (2025-09-22 02:34:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.990637623s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................X.X.......X..X......X.X.] duration: 2.732668675s FAIL (2025-09-22 02:34:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.73269045s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................X.X.......X..X......X.X] duration: 30.000893062s FAIL (2025-09-22 02:34:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000910488s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................XXXX.X....X......................] duration: 2.823173193s FAIL (2025-09-22 02:33:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.82320941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..........................XXXXXXX..X.X...XXXXXXX.X.X.X.XX...] duration: 2.363385021s FAIL (2025-09-22 02:33:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.363418546s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [...........................XXXXXXX..X.X...XXXXXXX.X.X.X.XX..] duration: 2.830263969s FAIL (2025-09-22 02:33:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.830330224s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................XXXX.X....X.....................] duration: 2.541228588s FAIL (2025-09-22 02:32:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.541262029s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................X.X.......X..X......X.] duration: 2.800742207s FAIL (2025-09-22 02:32:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.800768108s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................X.X.......X..X......X] duration: 30.001925518s FAIL (2025-09-22 02:32:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00196544s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [............................XXXXXXX..X.X...XXXXXXX.X.X.X.XX.] duration: 2.444928529s FAIL (2025-09-22 02:31:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.444961235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................XXXX.X....X....................] duration: 2.562286738s FAIL (2025-09-22 02:31:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.562377379s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.............................XXXXXXX..X.X...XXXXXXX.X.X.X.XX] duration: 30.218101637s FAIL (2025-09-22 02:31:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:30:42.935612090 (564h23m3.42804454s) [upload=496503]: Starting upload Sep 22 02:30:42.935707213 (564h23m3.42813965s) [upload=496503]: Done starting upload Sep 22 02:30:42.935908723 (564h23m3.42834117s) [upload=496503]: Uploading segments... Sep 22 02:30:42.937237020 (564h23m3.42966947s) [upload=496503]: Got next segment. Inline: false Sep 22 02:30:42.937331163 (564h23m3.429763609s) [upload=496503]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:30:42.937338108 (564h23m3.429770555s) [upload=496503]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:30:43.049335197 (564h23m3.541767771s) [seg_pos={0 0}, upload=496503]: Begin upload segment... Sep 22 02:30:43.049395010 (564h23m3.541827453s) [seg_pos={0 0}, upload=496503]: Done begin upload segment. Sep 22 02:30:43.049429038 (564h23m3.541861483s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049436345 (564h23m3.541868791s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049827183 (564h23m3.542259625s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.049840220 (564h23m3.542272664s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050324856 (564h23m3.542757302s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050621176 (564h23m3.543053618s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.050911128 (564h23m3.54334357s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051281605 (564h23m3.543714048s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051645875 (564h23m3.544078326s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.051968287 (564h23m3.54440073s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052258165 (564h23m3.544690609s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052532370 (564h23m3.544964816s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.052903004 (564h23m3.545335457s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053251407 (564h23m3.545683921s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.053563071 (564h23m3.545995507s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.057709349 (564h23m3.550141804s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Uploading piece... Sep 22 02:30:43.660353465 (564h23m4.15278591s) [upload=496503]: Next returned nil segment Sep 22 02:30:43.660378695 (564h23m4.152811149s) [upload=496503]: Waiting for error group managing segments... Sep 22 02:30:43.798242356 (564h23m4.290674812s) [noise=true , address=45.91.116.6:4975 , piece_id=VSWM2ERNC6P7SUVFY6C35VW64KQHTKLUNXT56KWKDHMZBY62ZC5A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800216247 (564h23m4.292648679s) [noise=true , address=45.91.116.12:4226 , piece_id=DGYLASNOIX2YBCAM2CNFG2QXJ7JRPSHVRM3KD6GMPRBPMVJP26VA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.800981914 (564h23m4.293414357s) [noise=true , address=77.90.31.15:4699 , piece_id=FOLGRBCVFEVQSZEBOP25L2KFCZ7BF7BC62C6QHSGQVQXYF5E5XUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.803128613 (564h23m4.295561048s) [noise=true , address=45.91.116.7:4011 , piece_id=C5HJGPFZFSAQWCUXJYI4QEPLGY2WLPABTJGJW7CNJSHQ2LBJG72Q, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.805491241 (564h23m4.297923686s) [noise=true , address=45.91.116.4:4937 , piece_id=M2YLYTYQIOB4HVEZRIG4SZCMIWOZKF2SJE56CSOP3ATPUUM3QOEA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817197466 (564h23m4.309629902s) [noise=true , address=77.90.31.15:4697 , piece_id=ICFBV5PNSJU7IRJ5BS7M7TID22TOLAEAT7DUIA34ET2H3B7QW3GQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.817417831 (564h23m4.309850266s) [noise=true , address=77.90.31.14:4663 , piece_id=E266WTNIS24APDXOQBZRTXX4ZDXGEZJ4G2V5JY7LUMU7YKWRHHAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820162262 (564h23m4.312594689s) [noise=true , address=45.91.116.8:4048 , piece_id=25ZHZJKLWBTCVZ5K7XJQLYJ7S5JIDC2RAH5AEMGTH5EKTYOVQACA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.820440448 (564h23m4.312872883s) [noise=true , address=45.91.116.10:4082 , piece_id=FMCI7MAHZGOWKIZXVQXGOQWRYTRK4DB52WQFTPNQLXP5WYMDKR3A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.828344388 (564h23m4.320776824s) [noise=true , address=45.91.116.6:4973 , piece_id=G4JPGOEP6RCO6GJRWNTIHWWZ7QRWZXBGR4MSQQ2DOB7TXUJOB3KA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:30:43.830886748 (564h23m4.323319197s) [noise=true , address=77.90.31.13:4626 , piece_id=2XJOTKT6O6RR6EN4TY6753KZGGKXOVCLCKBJHM3YPZUTRKECZVAA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: <nil> Sep 22 02:31:12.641124843 (564h23m33.133557276s) [noise=true , address=145.79.227.14:4839 , piece_id=AIQRFEFF7RYFQKNNNZP3EP7YRJJWNCVGZMNKIZZTZ5UHNTG33R6A, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: rpc: dial tcp 145.79.227.14:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641197622 (564h23m33.133630057s) [noise=true , address=145.79.227.2:4114 , piece_id=IBTBLA5VJS4MHM5LSFYBZ44O35PSJVLWEPCORDIXCIGWBTAMIGUA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641432201 (564h23m33.133864632s) [noise=true , address=145.79.227.16:4911 , piece_id=GFSLDZDOJU34SAANJHJOVE2EJH5G4MDMTAUNPRTJQOA5HZXPDRVA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641636455 (564h23m33.134068888s) [noise=true , address=145.79.227.7:4293 , piece_id=GSQFD6NGUNBUDMUHPIVU5RX3WLNQ5XNSWDKPLUAZZ3WYMMXMIDKA, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.641821062 (564h23m33.134253493s) [noise=true , address=145.79.227.6:4258 , piece_id=KNHEJC5BC73W2TXKNTYJAEQGKXUOEX2WSN2QGOD7T2K2Q3GSZRIQ, seg_pos={0 0}, upload=496503]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:31:12.642350097 (564h23m33.134782543s) [seg_pos={0 0}, upload=496503]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:31:12.856566764 (564h23m33.348999198s) [upload=496503]: Done uploading segments... Sep 22 02:31:12.856603777 (564h23m33.349036214s) [upload=496503]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.218129194s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................X.X.......X..X......] duration: 3.177061694s FAIL (2025-09-22 02:31:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.177230701s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................XXXX.X....X...................] duration: 2.619107133s FAIL (2025-09-22 02:30:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.619224025s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)