Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......X.X.......X..X......X.X.....X.X...XX....X..........X.] duration: 2.855797145s FAIL (2025-09-22 02:54:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.855829151s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [XXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X.....XX.] duration: 2.553265581s FAIL (2025-09-22 02:54:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:53:16.803730112 (564h45m37.296162545s) [upload=496844]: Starting upload Sep 22 02:53:16.803799753 (564h45m37.296232188s) [upload=496844]: Done starting upload Sep 22 02:53:16.803915386 (564h45m37.296347821s) [upload=496844]: Uploading segments... Sep 22 02:53:16.804081266 (564h45m37.296513698s) [upload=496844]: Got next segment. Inline: false Sep 22 02:53:16.804136100 (564h45m37.296568535s) [upload=496844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:53:16.804142089 (564h45m37.296574532s) [upload=496844]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:53:16.924693065 (564h45m37.417125514s) [seg_pos={0 0}, upload=496844]: Begin upload segment... Sep 22 02:53:16.924761853 (564h45m37.417194292s) [seg_pos={0 0}, upload=496844]: Done begin upload segment. Sep 22 02:53:16.924794582 (564h45m37.417227034s) [noise=true , address=45.91.116.9:4262 , piece_id=AN2YEHJFWO3GV36HVYEUEBYWX7APL4GI2JGZVLTVMQRZSR74AENA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.925328296 (564h45m37.417760741s) [noise=true , address=77.90.31.17:4770 , piece_id=OSZDK2HOGWIPYQHHY3AYZHTEPLJLKFJBGHVY5IMPEEQLL3BFN7PA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.925670244 (564h45m37.418102687s) [noise=true , address=45.91.116.14:4190 , piece_id=LRN66OHJO5ST7F57Q66SGTW5T7Q2N4IK52TZXYWA232BYL4DNSCQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926006094 (564h45m37.418438536s) [noise=true , address=45.91.116.8:4045 , piece_id=VQRXTUUXBGHHINC26X75NGG3OAGRSJOJ3Y65EAXB7ENUQWFE6WRQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926206123 (564h45m37.418638568s) [noise=true , address=77.90.31.17:4771 , piece_id=MVI3VRDUA5K5O4N5UP2DP5OLCCJC44P4RAR4FLXJ5ZS77D27KXGA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926834042 (564h45m37.419266491s) [noise=true , address=77.90.31.11:4555 , piece_id=BCTTNFWLDS3RA5XZRKJJIMRHGMXRFJMA2FUCB5EXSIVLRRZHESDA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927294956 (564h45m37.419727397s) [noise=true , address=45.91.116.6:4973 , piece_id=WUIM6UBQH7GEXD2BAVDDNYWF4GQFWWCL5MRXCKZ5UJS5W5634EBA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927573044 (564h45m37.420005486s) [noise=true , address=45.91.116.4:4937 , piece_id=HXJZJYSK3A2IAT3PTE5SA5UGEPCXATEJOKITCQ2D7CWTVPDGPEBA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927864772 (564h45m37.420297214s) [noise=true , address=45.91.116.5:4299 , piece_id=3IKQPV4BWIA2ON2GBOATUDK7LPZ3SVGIIV46QCSI5IGSO42SROPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928013610 (564h45m37.420446133s) [noise=true , address=45.91.116.10:4081 , piece_id=WDVMAL2L4I5MOUSECP6S4PXL3TK53SBRRLYAKWR2RXO7PWQLXJSA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928149679 (564h45m37.420582123s) [noise=true , address=77.90.31.19:4841 , piece_id=Z52MG5CYIAUXMV32PHBPLYKGNCSJJSEGAK4DHHN3LJMYFHOJIL5A, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928546383 (564h45m37.420978825s) [noise=true , address=145.79.227.11:4732 , piece_id=KD4CBSFZ7RLL3FBKZ4NQKLS7PXFNQKXXGAOKKG5MERP423UJHOPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928891033 (564h45m37.421323475s) [noise=true , address=145.79.227.4:4186 , piece_id=2LFV4O5E5ZHYN24SEDLHHO334CDAW4AW7PAN322QXT3H2SRFSLPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928931827 (564h45m37.421364267s) [noise=true , address=145.79.227.16:4909 , piece_id=2F6FGN2RYDM5IUBZ5AXMQE7RYK3NKBKLCMGJOQ474KGWBMMHK2RQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.929444932 (564h45m37.421877365s) [noise=true , address=45.91.116.11:4118 , piece_id=5K5IYEDQU3PSFWG2IWNKWFOHWMG6BAQCPLIJLDHQQWEJYRNZ2GGA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.929986031 (564h45m37.422418464s) [noise=true , address=145.79.227.18:4005 , piece_id=KXBNHWEZASVN6O4N76LRA5Z7EYGKSGJZWXX26FGJC5OZD5CVO6AA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:17.532291405 (564h45m38.024723835s) [upload=496844]: Next returned nil segment Sep 22 02:53:17.532308544 (564h45m38.024740987s) [upload=496844]: Waiting for error group managing segments... Sep 22 02:53:17.673641008 (564h45m38.166073489s) [noise=true , address=45.91.116.9:4262 , piece_id=AN2YEHJFWO3GV36HVYEUEBYWX7APL4GI2JGZVLTVMQRZSR74AENA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.677178652 (564h45m38.169611082s) [noise=true , address=45.91.116.10:4081 , piece_id=WDVMAL2L4I5MOUSECP6S4PXL3TK53SBRRLYAKWR2RXO7PWQLXJSA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.678967137 (564h45m38.171399573s) [noise=true , address=45.91.116.4:4937 , piece_id=HXJZJYSK3A2IAT3PTE5SA5UGEPCXATEJOKITCQ2D7CWTVPDGPEBA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.681878108 (564h45m38.174310536s) [noise=true , address=45.91.116.11:4118 , piece_id=5K5IYEDQU3PSFWG2IWNKWFOHWMG6BAQCPLIJLDHQQWEJYRNZ2GGA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.686890524 (564h45m38.179322957s) [noise=true , address=77.90.31.11:4555 , piece_id=BCTTNFWLDS3RA5XZRKJJIMRHGMXRFJMA2FUCB5EXSIVLRRZHESDA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.686940587 (564h45m38.179373029s) [noise=true , address=77.90.31.19:4841 , piece_id=Z52MG5CYIAUXMV32PHBPLYKGNCSJJSEGAK4DHHN3LJMYFHOJIL5A, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687424121 (564h45m38.179856556s) [noise=true , address=77.90.31.17:4770 , piece_id=OSZDK2HOGWIPYQHHY3AYZHTEPLJLKFJBGHVY5IMPEEQLL3BFN7PA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687535995 (564h45m38.179968424s) [noise=true , address=45.91.116.14:4190 , piece_id=LRN66OHJO5ST7F57Q66SGTW5T7Q2N4IK52TZXYWA232BYL4DNSCQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687981077 (564h45m38.180413511s) [noise=true , address=77.90.31.17:4771 , piece_id=MVI3VRDUA5K5O4N5UP2DP5OLCCJC44P4RAR4FLXJ5ZS77D27KXGA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.688037279 (564h45m38.180469715s) [noise=true , address=45.91.116.8:4045 , piece_id=VQRXTUUXBGHHINC26X75NGG3OAGRSJOJ3Y65EAXB7ENUQWFE6WRQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.690583072 (564h45m38.183015501s) [noise=true , address=45.91.116.6:4973 , piece_id=WUIM6UBQH7GEXD2BAVDDNYWF4GQFWWCL5MRXCKZ5UJS5W5634EBA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.697337949 (564h45m38.189770384s) [noise=true , address=45.91.116.5:4299 , piece_id=3IKQPV4BWIA2ON2GBOATUDK7LPZ3SVGIIV46QCSI5IGSO42SROPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.901510874 (564h45m38.393943323s) [noise=true , address=145.79.227.11:4732 , piece_id=KD4CBSFZ7RLL3FBKZ4NQKLS7PXFNQKXXGAOKKG5MERP423UJHOPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499213541 (564h46m6.991645996s) [noise=true , address=145.79.227.4:4186 , piece_id=2LFV4O5E5ZHYN24SEDLHHO334CDAW4AW7PAN322QXT3H2SRFSLPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499294035 (564h46m6.99172648s) [noise=true , address=145.79.227.16:4909 , piece_id=2F6FGN2RYDM5IUBZ5AXMQE7RYK3NKBKLCMGJOQ474KGWBMMHK2RQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499316904 (564h46m6.991749347s) [noise=true , address=145.79.227.18:4005 , piece_id=KXBNHWEZASVN6O4N76LRA5Z7EYGKSGJZWXX26FGJC5OZD5CVO6AA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499374361 (564h46m6.99180681s) [seg_pos={0 0}, upload=496844]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:53:46.611368399 (564h46m7.103800835s) [upload=496844]: Done uploading segments... Sep 22 02:53:46.611396228 (564h46m7.103828662s) [upload=496844]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.553304664s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........X.X.......X..X......X.X.....X.X...XX....X..........X] duration: 30.001989287s FAIL (2025-09-22 02:54:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002022558s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [XXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X.....XX] duration: 30.114037542s FAIL (2025-09-22 02:53:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:53:16.803730112 (564h45m37.296162545s) [upload=496844]: Starting upload Sep 22 02:53:16.803799753 (564h45m37.296232188s) [upload=496844]: Done starting upload Sep 22 02:53:16.803915386 (564h45m37.296347821s) [upload=496844]: Uploading segments... Sep 22 02:53:16.804081266 (564h45m37.296513698s) [upload=496844]: Got next segment. Inline: false Sep 22 02:53:16.804136100 (564h45m37.296568535s) [upload=496844]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:53:16.804142089 (564h45m37.296574532s) [upload=496844]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:53:16.924693065 (564h45m37.417125514s) [seg_pos={0 0}, upload=496844]: Begin upload segment... Sep 22 02:53:16.924761853 (564h45m37.417194292s) [seg_pos={0 0}, upload=496844]: Done begin upload segment. Sep 22 02:53:16.924794582 (564h45m37.417227034s) [noise=true , address=45.91.116.9:4262 , piece_id=AN2YEHJFWO3GV36HVYEUEBYWX7APL4GI2JGZVLTVMQRZSR74AENA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.925328296 (564h45m37.417760741s) [noise=true , address=77.90.31.17:4770 , piece_id=OSZDK2HOGWIPYQHHY3AYZHTEPLJLKFJBGHVY5IMPEEQLL3BFN7PA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.925670244 (564h45m37.418102687s) [noise=true , address=45.91.116.14:4190 , piece_id=LRN66OHJO5ST7F57Q66SGTW5T7Q2N4IK52TZXYWA232BYL4DNSCQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926006094 (564h45m37.418438536s) [noise=true , address=45.91.116.8:4045 , piece_id=VQRXTUUXBGHHINC26X75NGG3OAGRSJOJ3Y65EAXB7ENUQWFE6WRQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926206123 (564h45m37.418638568s) [noise=true , address=77.90.31.17:4771 , piece_id=MVI3VRDUA5K5O4N5UP2DP5OLCCJC44P4RAR4FLXJ5ZS77D27KXGA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.926834042 (564h45m37.419266491s) [noise=true , address=77.90.31.11:4555 , piece_id=BCTTNFWLDS3RA5XZRKJJIMRHGMXRFJMA2FUCB5EXSIVLRRZHESDA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927294956 (564h45m37.419727397s) [noise=true , address=45.91.116.6:4973 , piece_id=WUIM6UBQH7GEXD2BAVDDNYWF4GQFWWCL5MRXCKZ5UJS5W5634EBA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927573044 (564h45m37.420005486s) [noise=true , address=45.91.116.4:4937 , piece_id=HXJZJYSK3A2IAT3PTE5SA5UGEPCXATEJOKITCQ2D7CWTVPDGPEBA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.927864772 (564h45m37.420297214s) [noise=true , address=45.91.116.5:4299 , piece_id=3IKQPV4BWIA2ON2GBOATUDK7LPZ3SVGIIV46QCSI5IGSO42SROPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928013610 (564h45m37.420446133s) [noise=true , address=45.91.116.10:4081 , piece_id=WDVMAL2L4I5MOUSECP6S4PXL3TK53SBRRLYAKWR2RXO7PWQLXJSA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928149679 (564h45m37.420582123s) [noise=true , address=77.90.31.19:4841 , piece_id=Z52MG5CYIAUXMV32PHBPLYKGNCSJJSEGAK4DHHN3LJMYFHOJIL5A, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928546383 (564h45m37.420978825s) [noise=true , address=145.79.227.11:4732 , piece_id=KD4CBSFZ7RLL3FBKZ4NQKLS7PXFNQKXXGAOKKG5MERP423UJHOPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928891033 (564h45m37.421323475s) [noise=true , address=145.79.227.4:4186 , piece_id=2LFV4O5E5ZHYN24SEDLHHO334CDAW4AW7PAN322QXT3H2SRFSLPA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.928931827 (564h45m37.421364267s) [noise=true , address=145.79.227.16:4909 , piece_id=2F6FGN2RYDM5IUBZ5AXMQE7RYK3NKBKLCMGJOQ474KGWBMMHK2RQ, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.929444932 (564h45m37.421877365s) [noise=true , address=45.91.116.11:4118 , piece_id=5K5IYEDQU3PSFWG2IWNKWFOHWMG6BAQCPLIJLDHQQWEJYRNZ2GGA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:16.929986031 (564h45m37.422418464s) [noise=true , address=145.79.227.18:4005 , piece_id=KXBNHWEZASVN6O4N76LRA5Z7EYGKSGJZWXX26FGJC5OZD5CVO6AA, seg_pos={0 0}, upload=496844]: Uploading piece... Sep 22 02:53:17.532291405 (564h45m38.024723835s) [upload=496844]: Next returned nil segment Sep 22 02:53:17.532308544 (564h45m38.024740987s) [upload=496844]: Waiting for error group managing segments... Sep 22 02:53:17.673641008 (564h45m38.166073489s) [noise=true , address=45.91.116.9:4262 , piece_id=AN2YEHJFWO3GV36HVYEUEBYWX7APL4GI2JGZVLTVMQRZSR74AENA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.677178652 (564h45m38.169611082s) [noise=true , address=45.91.116.10:4081 , piece_id=WDVMAL2L4I5MOUSECP6S4PXL3TK53SBRRLYAKWR2RXO7PWQLXJSA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.678967137 (564h45m38.171399573s) [noise=true , address=45.91.116.4:4937 , piece_id=HXJZJYSK3A2IAT3PTE5SA5UGEPCXATEJOKITCQ2D7CWTVPDGPEBA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.681878108 (564h45m38.174310536s) [noise=true , address=45.91.116.11:4118 , piece_id=5K5IYEDQU3PSFWG2IWNKWFOHWMG6BAQCPLIJLDHQQWEJYRNZ2GGA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.686890524 (564h45m38.179322957s) [noise=true , address=77.90.31.11:4555 , piece_id=BCTTNFWLDS3RA5XZRKJJIMRHGMXRFJMA2FUCB5EXSIVLRRZHESDA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.686940587 (564h45m38.179373029s) [noise=true , address=77.90.31.19:4841 , piece_id=Z52MG5CYIAUXMV32PHBPLYKGNCSJJSEGAK4DHHN3LJMYFHOJIL5A, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687424121 (564h45m38.179856556s) [noise=true , address=77.90.31.17:4770 , piece_id=OSZDK2HOGWIPYQHHY3AYZHTEPLJLKFJBGHVY5IMPEEQLL3BFN7PA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687535995 (564h45m38.179968424s) [noise=true , address=45.91.116.14:4190 , piece_id=LRN66OHJO5ST7F57Q66SGTW5T7Q2N4IK52TZXYWA232BYL4DNSCQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.687981077 (564h45m38.180413511s) [noise=true , address=77.90.31.17:4771 , piece_id=MVI3VRDUA5K5O4N5UP2DP5OLCCJC44P4RAR4FLXJ5ZS77D27KXGA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.688037279 (564h45m38.180469715s) [noise=true , address=45.91.116.8:4045 , piece_id=VQRXTUUXBGHHINC26X75NGG3OAGRSJOJ3Y65EAXB7ENUQWFE6WRQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.690583072 (564h45m38.183015501s) [noise=true , address=45.91.116.6:4973 , piece_id=WUIM6UBQH7GEXD2BAVDDNYWF4GQFWWCL5MRXCKZ5UJS5W5634EBA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.697337949 (564h45m38.189770384s) [noise=true , address=45.91.116.5:4299 , piece_id=3IKQPV4BWIA2ON2GBOATUDK7LPZ3SVGIIV46QCSI5IGSO42SROPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: <nil> Sep 22 02:53:17.901510874 (564h45m38.393943323s) [noise=true , address=145.79.227.11:4732 , piece_id=KD4CBSFZ7RLL3FBKZ4NQKLS7PXFNQKXXGAOKKG5MERP423UJHOPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499213541 (564h46m6.991645996s) [noise=true , address=145.79.227.4:4186 , piece_id=2LFV4O5E5ZHYN24SEDLHHO334CDAW4AW7PAN322QXT3H2SRFSLPA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499294035 (564h46m6.99172648s) [noise=true , address=145.79.227.16:4909 , piece_id=2F6FGN2RYDM5IUBZ5AXMQE7RYK3NKBKLCMGJOQ474KGWBMMHK2RQ, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499316904 (564h46m6.991749347s) [noise=true , address=145.79.227.18:4005 , piece_id=KXBNHWEZASVN6O4N76LRA5Z7EYGKSGJZWXX26FGJC5OZD5CVO6AA, seg_pos={0 0}, upload=496844]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:46.499374361 (564h46m6.99180681s) [seg_pos={0 0}, upload=496844]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:53:46.611368399 (564h46m7.103800835s) [upload=496844]: Done uploading segments... Sep 22 02:53:46.611396228 (564h46m7.103828662s) [upload=496844]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.114055174s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.X....X....................................................] duration: 2.580943487s FAIL (2025-09-22 02:53:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.581001051s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [XX.X....X...................................................] duration: 2.509288782s FAIL (2025-09-22 02:53:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.509349304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X.....X] duration: 30.116111169s FAIL (2025-09-22 02:53:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:52:46.525991798 (564h45m7.018424229s) [upload=496833]: Starting upload Sep 22 02:52:46.526064102 (564h45m7.018496547s) [upload=496833]: Done starting upload Sep 22 02:52:46.526215221 (564h45m7.018647655s) [upload=496833]: Uploading segments... Sep 22 02:52:46.526441803 (564h45m7.018874247s) [upload=496833]: Got next segment. Inline: false Sep 22 02:52:46.526489824 (564h45m7.018922266s) [upload=496833]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:52:46.526497372 (564h45m7.018929801s) [upload=496833]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:52:46.638913325 (564h45m7.131345755s) [seg_pos={0 0}, upload=496833]: Begin upload segment... Sep 22 02:52:46.638972883 (564h45m7.131405311s) [seg_pos={0 0}, upload=496833]: Done begin upload segment. Sep 22 02:52:46.638997942 (564h45m7.131430385s) [noise=true , address=45.91.116.12:4225 , piece_id=IQ257HGG24STOXT55X54Z5VKFE7YNN37WGL2TTTM4WFSCMUMDRDA, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.639027165 (564h45m7.131459613s) [noise=true , address=145.79.227.5:4222 , piece_id=GPRZLMVYROXQJYZ5AOI3D4Q6237IJMO74RHOSDJS2Z65SQBLORNQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.639419151 (564h45m7.131851597s) [noise=true , address=145.79.227.11:4729 , piece_id=WWT37RT4SMOXU6JRBMSN4WTWUYBT5NCLRKZ6EU3CIMHAX227SDVA, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.639696618 (564h45m7.132129061s) [noise=true , address=145.79.227.6:4258 , piece_id=PFOLPRIHWZJLRHHGFCO3OOMPH37DR3PBKA5A6GL76HIDX7IBQJIQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.640008719 (564h45m7.132441162s) [noise=true , address=145.79.227.13:4801 , piece_id=RQBQ6FFK4XWQCEYDHC5C7AKVKNRYNLQKAL7CTYCKQX4I5QUQ6F6Q, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.640289702 (564h45m7.132722147s) [noise=true , address=145.79.227.18:4005 , piece_id=5OB3JHNKKSL73EDE4XMZRWOXCXHHZJ2S2CNXXSQGBTCYKELWL7KA, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.640653170 (564h45m7.133085619s) [noise=true , address=45.91.116.14:4189 , piece_id=XK4FA45DOQA5YO42XZM7OXVQHOJMZ5VDYLJ3WX6HWGSRAYD3VDQA, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.641009407 (564h45m7.133441853s) [noise=true , address=77.90.31.19:4841 , piece_id=23G273OAMXQQC3BDKQBSY7X2WB5ZZMGGYEPB2NBTLS7CXLBE7C4A, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.641318455 (564h45m7.133750901s) [noise=true , address=77.90.31.15:4697 , piece_id=BCMVIINNHMMTE6U2SA5VX2KYQB4EAP2F46XE766KBYMWMZ2CVS2A, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.643272908 (564h45m7.135705351s) [noise=true , address=77.90.31.21:4878 , piece_id=VRE6HZYPCYYUQASO3C22WGPWQTQPONQV2IIAUBGQNR4FITEYJKLA, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.643801108 (564h45m7.13623355s) [noise=true , address=45.91.116.7:4010 , piece_id=DCRJBZ6R3KQ2PTPOLGC6YYFFWD4J2G4YJAO25KL5X2THQ4PAJYSQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.644234808 (564h45m7.136667251s) [noise=true , address=145.79.227.15:4875 , piece_id=EX6EOJQIOSEMOE7MP7HP2T4YMZEAE4DRNX7XRSOQLVMO6FBIR2MQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.644652077 (564h45m7.137084521s) [noise=true , address=45.91.116.10:4083 , piece_id=B6IHKOWGIH6BAOBRYNTES4S46HJSYGC3N57VQTEGONJJMZZGXI5A, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.645105185 (564h45m7.137537627s) [noise=true , address=45.91.116.5:4298 , piece_id=RXFUJKIGQISWRODGB6HP7RWXCKEPZTMYW73I5YGVOWPU2YZM3ETQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.646589842 (564h45m7.139022287s) [noise=true , address=45.91.116.13:4155 , piece_id=J5NATRYJGDCKG4FW7IBZGKOFDYRBJZBQS4FENTBKL7IUT7BPPHKQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:46.647113719 (564h45m7.13954617s) [noise=true , address=77.90.31.10:4520 , piece_id=K4FQE5K5TYFCZ47JGXPBCKHX42P4WNXS4G7QSX562NJ7EBAQXRAQ, seg_pos={0 0}, upload=496833]: Uploading piece... Sep 22 02:52:47.261479471 (564h45m7.75391191s) [upload=496833]: Next returned nil segment Sep 22 02:52:47.261554351 (564h45m7.753986823s) [upload=496833]: Waiting for error group managing segments... Sep 22 02:52:47.391229173 (564h45m7.883661628s) [noise=true , address=45.91.116.7:4010 , piece_id=DCRJBZ6R3KQ2PTPOLGC6YYFFWD4J2G4YJAO25KL5X2THQ4PAJYSQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.392826511 (564h45m7.885258943s) [noise=true , address=77.90.31.19:4841 , piece_id=23G273OAMXQQC3BDKQBSY7X2WB5ZZMGGYEPB2NBTLS7CXLBE7C4A, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.393698816 (564h45m7.886131258s) [noise=true , address=45.91.116.13:4155 , piece_id=J5NATRYJGDCKG4FW7IBZGKOFDYRBJZBQS4FENTBKL7IUT7BPPHKQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.394499443 (564h45m7.886931887s) [noise=true , address=77.90.31.21:4878 , piece_id=VRE6HZYPCYYUQASO3C22WGPWQTQPONQV2IIAUBGQNR4FITEYJKLA, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.395984248 (564h45m7.888416682s) [noise=true , address=45.91.116.5:4298 , piece_id=RXFUJKIGQISWRODGB6HP7RWXCKEPZTMYW73I5YGVOWPU2YZM3ETQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.396351900 (564h45m7.88878433s) [noise=true , address=77.90.31.15:4697 , piece_id=BCMVIINNHMMTE6U2SA5VX2KYQB4EAP2F46XE766KBYMWMZ2CVS2A, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.404086880 (564h45m7.896519318s) [noise=true , address=45.91.116.10:4083 , piece_id=B6IHKOWGIH6BAOBRYNTES4S46HJSYGC3N57VQTEGONJJMZZGXI5A, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.405624841 (564h45m7.898057271s) [noise=true , address=45.91.116.14:4189 , piece_id=XK4FA45DOQA5YO42XZM7OXVQHOJMZ5VDYLJ3WX6HWGSRAYD3VDQA, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.407379019 (564h45m7.899811462s) [noise=true , address=45.91.116.12:4225 , piece_id=IQ257HGG24STOXT55X54Z5VKFE7YNN37WGL2TTTM4WFSCMUMDRDA, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:52:47.412001122 (564h45m7.904433558s) [noise=true , address=77.90.31.10:4520 , piece_id=K4FQE5K5TYFCZ47JGXPBCKHX42P4WNXS4G7QSX562NJ7EBAQXRAQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: <nil> Sep 22 02:53:16.243788897 (564h45m36.736221331s) [noise=true , address=145.79.227.5:4222 , piece_id=GPRZLMVYROXQJYZ5AOI3D4Q6237IJMO74RHOSDJS2Z65SQBLORNQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:16.243851825 (564h45m36.73628426s) [noise=true , address=145.79.227.18:4005 , piece_id=5OB3JHNKKSL73EDE4XMZRWOXCXHHZJ2S2CNXXSQGBTCYKELWL7KA, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:16.243895516 (564h45m36.736327958s) [noise=true , address=145.79.227.11:4729 , piece_id=WWT37RT4SMOXU6JRBMSN4WTWUYBT5NCLRKZ6EU3CIMHAX227SDVA, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:16.243938325 (564h45m36.736370768s) [noise=true , address=145.79.227.6:4258 , piece_id=PFOLPRIHWZJLRHHGFCO3OOMPH37DR3PBKA5A6GL76HIDX7IBQJIQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:16.243982004 (564h45m36.736414447s) [noise=true , address=145.79.227.13:4801 , piece_id=RQBQ6FFK4XWQCEYDHC5C7AKVKNRYNLQKAL7CTYCKQX4I5QUQ6F6Q, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:53:16.244124190 (564h45m36.736556632s) [noise=true , address=145.79.227.15:4875 , piece_id=EX6EOJQIOSEMOE7MP7HP2T4YMZEAE4DRNX7XRSOQLVMO6FBIR2MQ, seg_pos={0 0}, upload=496833]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:53:16.244143107 (564h45m36.736575552s) [seg_pos={0 0}, upload=496833]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:53:16.358409801 (564h45m36.85084223s) [upload=496833]: Done uploading segments... Sep 22 02:53:16.358437671 (564h45m36.850870113s) [upload=496833]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.11614688s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........X.X.......X..X......X.X.....X.X...XX....X..........] duration: 2.739583431s FAIL (2025-09-22 02:53:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.739612577s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXX.X....X..................................................] duration: 2.703196391s FAIL (2025-09-22 02:52:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.703229797s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........X.X.......X..X......X.X.....X.X...XX....X.........] duration: 2.826207827s FAIL (2025-09-22 02:52:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.826235947s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [.XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X.....] duration: 2.466575837s FAIL (2025-09-22 02:52:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.466611449s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXXX.X....X.................................................] duration: 2.593656314s FAIL (2025-09-22 02:51:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.593688926s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........X.X.......X..X......X.X.....X.X...XX....X........] duration: 2.96774612s FAIL (2025-09-22 02:51:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.967770755s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [..XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X....] duration: 2.775446759s FAIL (2025-09-22 02:51:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.77548026s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............X.X.......X..X......X.X.....X.X...XX....X.......] duration: 2.755017282s FAIL (2025-09-22 02:50:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.755048416s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.XXXX.X....X................................................] duration: 2.624426058s FAIL (2025-09-22 02:50:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.62445872s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [...XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X...] duration: 2.585032027s FAIL (2025-09-22 02:50:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.585124848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [....XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X..] duration: 2.6860792s FAIL (2025-09-22 02:50:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.686113568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..XXXX.X....X...............................................] duration: 2.562210231s FAIL (2025-09-22 02:50:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.562265855s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [.....XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X.] duration: 2.505117945s FAIL (2025-09-22 02:49:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.505152429s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............X.X.......X..X......X.X.....X.X...XX....X......] duration: 2.718175851s FAIL (2025-09-22 02:49:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.718207773s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...XXXX.X....X..............................................] duration: 3.069175934s FAIL (2025-09-22 02:49:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 3.069205912s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [......XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....X] duration: 30.113656335s FAIL (2025-09-22 02:49:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:49:05.085962621 (564h41m25.578395055s) [upload=496781]: Starting upload Sep 22 02:49:05.086140282 (564h41m25.578572724s) [upload=496781]: Done starting upload Sep 22 02:49:05.086240609 (564h41m25.578673046s) [upload=496781]: Uploading segments... Sep 22 02:49:05.086692301 (564h41m25.579124743s) [upload=496781]: Got next segment. Inline: false Sep 22 02:49:05.086786060 (564h41m25.579218488s) [upload=496781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:49:05.086794251 (564h41m25.579226698s) [upload=496781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:49:05.204305195 (564h41m25.696737627s) [seg_pos={0 0}, upload=496781]: Begin upload segment... Sep 22 02:49:05.204356829 (564h41m25.696789261s) [seg_pos={0 0}, upload=496781]: Done begin upload segment. Sep 22 02:49:05.204386463 (564h41m25.696818905s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204421070 (564h41m25.696853521s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204789126 (564h41m25.697221571s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.204967907 (564h41m25.697400361s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205105252 (564h41m25.697537705s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205444036 (564h41m25.697876468s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205510840 (564h41m25.697943292s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.205833391 (564h41m25.698265836s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206317736 (564h41m25.698750181s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.206662013 (564h41m25.699094462s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207002950 (564h41m25.699435392s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207015263 (564h41m25.69944772s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207343770 (564h41m25.699776222s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207621351 (564h41m25.700053795s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.207901423 (564h41m25.700333872s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.208221348 (564h41m25.700653793s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Uploading piece... Sep 22 02:49:05.820125819 (564h41m26.312558254s) [upload=496781]: Next returned nil segment Sep 22 02:49:05.820571715 (564h41m26.313004163s) [upload=496781]: Waiting for error group managing segments... Sep 22 02:49:05.958761934 (564h41m26.45119437s) [noise=true , address=77.90.31.10:4518 , piece_id=HVCSAXHHZ5D66MJ5NZLFHUMSOYG5KO2RCRWLCPETEDLTRRGXFGKQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.967100493 (564h41m26.459532928s) [noise=true , address=45.91.116.12:4227 , piece_id=X4MLO5JTJ46LSCSSVKDIJ4GRHLD25OG3FMQGQSYS5ZG7JMIGRCDQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.969201921 (564h41m26.461634356s) [noise=true , address=77.90.31.11:4554 , piece_id=OP3AYDKHVU4KCKCALCWGQTHO55FXIRPE3RXNBNJ4HBAK2SVG5AMQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970120880 (564h41m26.462553313s) [noise=true , address=77.90.31.18:4807 , piece_id=EEKIAGSK4SDWCXVVGH5E2PCPYYPFBYSMJEZKELZHQDM2WNA34Y4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.970646400 (564h41m26.463078828s) [noise=true , address=45.91.116.10:4082 , piece_id=KRNW43EEGUUZPJBI42BDT3GUVPYGPQ6AAQGHJAAFF6TPUYYW7SKA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.976606511 (564h41m26.469038946s) [noise=true , address=45.91.116.11:4118 , piece_id=UVSGVZSXFNXK6CDFNGCBSPF65GJ3GLMV3CNHJOSUV4HPBEBFVRZQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977158481 (564h41m26.469590943s) [noise=true , address=77.90.31.16:4735 , piece_id=QEIEEBXCSVXQCG5CAYWOJUXY7RDFLMYEP42NP5PYJJU32GNOPKOQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.977835284 (564h41m26.470267721s) [noise=true , address=45.91.116.14:4191 , piece_id=D357SZZMP3WHZE3YUVFUZBV6ANNBGSE3O6V6AKN7RCJRQX2RDTQQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980275742 (564h41m26.472708179s) [noise=true , address=45.91.116.9:4261 , piece_id=MVNTEDDZ5LYV7KNAMPGSLRTQB5TO7VICATD5ZBKEITDQPY3S54ZA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.980374815 (564h41m26.472807249s) [noise=true , address=45.91.116.8:4047 , piece_id=TZJ3B4WD3VMCACUGFMBV453CWQKXEVAKWSB4AEFWRHBKO4DYH6VQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981413898 (564h41m26.473846343s) [noise=true , address=77.90.31.21:4879 , piece_id=TXYW7AG6IXHYGOGX5PYT3K6SCZW7W5C6SXNKIAETUA6V2PAQ2OYA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:05.981569454 (564h41m26.47400189s) [noise=true , address=77.90.31.15:4699 , piece_id=YPDNAALKSXB6LIKFAMPJ2ORARGYNRWMZDG5TUTYQ7EMDPYJTLX4Q, seg_pos={0 0}, upload=496781]: Done uploading piece. err: <nil> Sep 22 02:49:34.779181706 (564h41m55.271614155s) [noise=true , address=145.79.227.17:4972 , piece_id=VY2BAT54M74LY7DOF74GGLS3F5D5E2TE6HKEPZU3QPJDXZQPL5CA, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779219456 (564h41m55.27165189s) [noise=true , address=145.79.227.19:4078 , piece_id=YNQY2YRQ7Q3I4VA5YE2KDF6Z6B4E3IQEVTP3M7IGEGXXEG52DTFQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: rpc: dial tcp 145.79.227.19:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779262230 (564h41m55.271694676s) [noise=true , address=145.79.227.4:4185 , piece_id=3BNKTYGMJ3TMDUKLSEXZIGRTM3ULNJUYL5ZHPHBW6O3MQ7RXN2RQ, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779288536 (564h41m55.271720972s) [noise=true , address=145.79.227.4:4186 , piece_id=5UGGCVZVLML4NENXKRMJVU6ZYSQ65SDXS6WKCNZA7P237OQIFF2A, seg_pos={0 0}, upload=496781]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:49:34.779314527 (564h41m55.271746964s) [seg_pos={0 0}, upload=496781]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:49:34.890859835 (564h41m55.383292281s) [upload=496781]: Done uploading segments... Sep 22 02:49:34.890893892 (564h41m55.383326339s) [upload=496781]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.113684744s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [....XXXX.X....X.............................................] duration: 2.50401971s FAIL (2025-09-22 02:49:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.504052921s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....XXXX.X....X............................................] duration: 2.524790972s FAIL (2025-09-22 02:49:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.524826903s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..............X.X.......X..X......X.X.....X.X...XX....X.....] duration: 2.77376769s FAIL (2025-09-22 02:48:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.773794151s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.......XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X....] duration: 2.445115586s FAIL (2025-09-22 02:48:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:45:48.123977404 (564h38m8.616409847s) [upload=496726]: Starting upload Sep 22 02:45:48.124048430 (564h38m8.616480865s) [upload=496726]: Done starting upload Sep 22 02:45:48.124142375 (564h38m8.616574808s) [upload=496726]: Uploading segments... Sep 22 02:45:48.124434847 (564h38m8.61686728s) [upload=496726]: Got next segment. Inline: false Sep 22 02:45:48.124485207 (564h38m8.616917657s) [upload=496726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:45:48.124491491 (564h38m8.616923954s) [upload=496726]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:45:48.237565267 (564h38m8.729997699s) [seg_pos={0 0}, upload=496726]: Begin upload segment... Sep 22 02:45:48.237618508 (564h38m8.730050944s) [seg_pos={0 0}, upload=496726]: Done begin upload segment. Sep 22 02:45:48.237645612 (564h38m8.730078054s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.237684929 (564h38m8.73011738s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238207044 (564h38m8.730639491s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238531941 (564h38m8.730964386s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238846154 (564h38m8.731278605s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239244187 (564h38m8.731676641s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239755762 (564h38m8.732188207s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239801248 (564h38m8.732233701s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240152427 (564h38m8.732584878s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240192870 (564h38m8.732625325s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240393062 (564h38m8.73282551s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240613542 (564h38m8.733045988s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240645601 (564h38m8.733078053s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240834348 (564h38m8.733266793s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241079643 (564h38m8.733512093s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241433198 (564h38m8.733865646s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.844540185 (564h38m9.33697262s) [upload=496726]: Next returned nil segment Sep 22 02:45:48.845994696 (564h38m9.338427142s) [upload=496726]: Waiting for error group managing segments... Sep 22 02:45:48.982962267 (564h38m9.475394716s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.983976197 (564h38m9.476408635s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.986859791 (564h38m9.47929222s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987790541 (564h38m9.480222978s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987885955 (564h38m9.48031839s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988406341 (564h38m9.480838769s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988516453 (564h38m9.48094889s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.994795403 (564h38m9.487227836s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.995545605 (564h38m9.487978038s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.998323112 (564h38m9.490755548s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:49.003694081 (564h38m9.496126531s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:50.247858948 (564h38m10.740291397s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: connect: no route to host; conn closed: op Sep 22 02:46:17.848383509 (564h38m38.340815937s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.848632149 (564h38m38.341064593s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849472038 (564h38m38.341904483s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849621286 (564h38m38.342053718s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849657569 (564h38m38.342090015s) [seg_pos={0 0}, upload=496726]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:46:17.949774543 (564h38m38.442206977s) [upload=496726]: Done uploading segments... Sep 22 02:46:17.949816811 (564h38m38.442249241s) [upload=496726]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.445149555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [......XXXX.X....X...........................................] duration: 2.755190302s FAIL (2025-09-22 02:48:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.755221408s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......XXXX.X....X..........................................] duration: 2.55993518s FAIL (2025-09-22 02:47:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.559978437s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...............X.X.......X..X......X.X.....X.X...XX....X....] duration: 2.750086597s FAIL (2025-09-22 02:47:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.750123648s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [........XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X...] duration: 2.506288228s FAIL (2025-09-22 02:47:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:45:48.123977404 (564h38m8.616409847s) [upload=496726]: Starting upload Sep 22 02:45:48.124048430 (564h38m8.616480865s) [upload=496726]: Done starting upload Sep 22 02:45:48.124142375 (564h38m8.616574808s) [upload=496726]: Uploading segments... Sep 22 02:45:48.124434847 (564h38m8.61686728s) [upload=496726]: Got next segment. Inline: false Sep 22 02:45:48.124485207 (564h38m8.616917657s) [upload=496726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:45:48.124491491 (564h38m8.616923954s) [upload=496726]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:45:48.237565267 (564h38m8.729997699s) [seg_pos={0 0}, upload=496726]: Begin upload segment... Sep 22 02:45:48.237618508 (564h38m8.730050944s) [seg_pos={0 0}, upload=496726]: Done begin upload segment. Sep 22 02:45:48.237645612 (564h38m8.730078054s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.237684929 (564h38m8.73011738s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238207044 (564h38m8.730639491s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238531941 (564h38m8.730964386s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238846154 (564h38m8.731278605s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239244187 (564h38m8.731676641s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239755762 (564h38m8.732188207s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239801248 (564h38m8.732233701s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240152427 (564h38m8.732584878s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240192870 (564h38m8.732625325s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240393062 (564h38m8.73282551s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240613542 (564h38m8.733045988s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240645601 (564h38m8.733078053s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240834348 (564h38m8.733266793s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241079643 (564h38m8.733512093s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241433198 (564h38m8.733865646s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.844540185 (564h38m9.33697262s) [upload=496726]: Next returned nil segment Sep 22 02:45:48.845994696 (564h38m9.338427142s) [upload=496726]: Waiting for error group managing segments... Sep 22 02:45:48.982962267 (564h38m9.475394716s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.983976197 (564h38m9.476408635s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.986859791 (564h38m9.47929222s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987790541 (564h38m9.480222978s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987885955 (564h38m9.48031839s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988406341 (564h38m9.480838769s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988516453 (564h38m9.48094889s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.994795403 (564h38m9.487227836s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.995545605 (564h38m9.487978038s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.998323112 (564h38m9.490755548s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:49.003694081 (564h38m9.496126531s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:50.247858948 (564h38m10.740291397s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: connect: no route to host; conn closed: op Sep 22 02:46:17.848383509 (564h38m38.340815937s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.848632149 (564h38m38.341064593s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849472038 (564h38m38.341904483s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849621286 (564h38m38.342053718s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849657569 (564h38m38.342090015s) [seg_pos={0 0}, upload=496726]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:46:17.949774543 (564h38m38.442206977s) [upload=496726]: Done uploading segments... Sep 22 02:46:17.949816811 (564h38m38.442249241s) [upload=496726]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.506322184s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................X.X.......X..X......X.X.....X.X...XX....X...] duration: 2.937356873s FAIL (2025-09-22 02:47:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.937386953s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [........XXXX.X....X.........................................] duration: 2.46583998s FAIL (2025-09-22 02:47:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.465875214s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........XXXX.X....X........................................] duration: 2.628732253s FAIL (2025-09-22 02:46:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.628766353s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.........XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X..] duration: 2.55798924s FAIL (2025-09-22 02:46:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:45:48.123977404 (564h38m8.616409847s) [upload=496726]: Starting upload Sep 22 02:45:48.124048430 (564h38m8.616480865s) [upload=496726]: Done starting upload Sep 22 02:45:48.124142375 (564h38m8.616574808s) [upload=496726]: Uploading segments... Sep 22 02:45:48.124434847 (564h38m8.61686728s) [upload=496726]: Got next segment. Inline: false Sep 22 02:45:48.124485207 (564h38m8.616917657s) [upload=496726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:45:48.124491491 (564h38m8.616923954s) [upload=496726]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:45:48.237565267 (564h38m8.729997699s) [seg_pos={0 0}, upload=496726]: Begin upload segment... Sep 22 02:45:48.237618508 (564h38m8.730050944s) [seg_pos={0 0}, upload=496726]: Done begin upload segment. Sep 22 02:45:48.237645612 (564h38m8.730078054s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.237684929 (564h38m8.73011738s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238207044 (564h38m8.730639491s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238531941 (564h38m8.730964386s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238846154 (564h38m8.731278605s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239244187 (564h38m8.731676641s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239755762 (564h38m8.732188207s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239801248 (564h38m8.732233701s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240152427 (564h38m8.732584878s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240192870 (564h38m8.732625325s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240393062 (564h38m8.73282551s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240613542 (564h38m8.733045988s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240645601 (564h38m8.733078053s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240834348 (564h38m8.733266793s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241079643 (564h38m8.733512093s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241433198 (564h38m8.733865646s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.844540185 (564h38m9.33697262s) [upload=496726]: Next returned nil segment Sep 22 02:45:48.845994696 (564h38m9.338427142s) [upload=496726]: Waiting for error group managing segments... Sep 22 02:45:48.982962267 (564h38m9.475394716s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.983976197 (564h38m9.476408635s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.986859791 (564h38m9.47929222s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987790541 (564h38m9.480222978s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987885955 (564h38m9.48031839s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988406341 (564h38m9.480838769s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988516453 (564h38m9.48094889s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.994795403 (564h38m9.487227836s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.995545605 (564h38m9.487978038s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.998323112 (564h38m9.490755548s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:49.003694081 (564h38m9.496126531s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:50.247858948 (564h38m10.740291397s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: connect: no route to host; conn closed: op Sep 22 02:46:17.848383509 (564h38m38.340815937s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.848632149 (564h38m38.341064593s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849472038 (564h38m38.341904483s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849621286 (564h38m38.342053718s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849657569 (564h38m38.342090015s) [seg_pos={0 0}, upload=496726]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:46:17.949774543 (564h38m38.442206977s) [upload=496726]: Done uploading segments... Sep 22 02:46:17.949816811 (564h38m38.442249241s) [upload=496726]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.55802454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........XXXX.X....X.......................................] duration: 2.513880363s FAIL (2025-09-22 02:46:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.513911157s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [..........XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X.] duration: 2.360248979s FAIL (2025-09-22 02:46:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:45:48.123977404 (564h38m8.616409847s) [upload=496726]: Starting upload Sep 22 02:45:48.124048430 (564h38m8.616480865s) [upload=496726]: Done starting upload Sep 22 02:45:48.124142375 (564h38m8.616574808s) [upload=496726]: Uploading segments... Sep 22 02:45:48.124434847 (564h38m8.61686728s) [upload=496726]: Got next segment. Inline: false Sep 22 02:45:48.124485207 (564h38m8.616917657s) [upload=496726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:45:48.124491491 (564h38m8.616923954s) [upload=496726]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:45:48.237565267 (564h38m8.729997699s) [seg_pos={0 0}, upload=496726]: Begin upload segment... Sep 22 02:45:48.237618508 (564h38m8.730050944s) [seg_pos={0 0}, upload=496726]: Done begin upload segment. Sep 22 02:45:48.237645612 (564h38m8.730078054s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.237684929 (564h38m8.73011738s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238207044 (564h38m8.730639491s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238531941 (564h38m8.730964386s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238846154 (564h38m8.731278605s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239244187 (564h38m8.731676641s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239755762 (564h38m8.732188207s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239801248 (564h38m8.732233701s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240152427 (564h38m8.732584878s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240192870 (564h38m8.732625325s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240393062 (564h38m8.73282551s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240613542 (564h38m8.733045988s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240645601 (564h38m8.733078053s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240834348 (564h38m8.733266793s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241079643 (564h38m8.733512093s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241433198 (564h38m8.733865646s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.844540185 (564h38m9.33697262s) [upload=496726]: Next returned nil segment Sep 22 02:45:48.845994696 (564h38m9.338427142s) [upload=496726]: Waiting for error group managing segments... Sep 22 02:45:48.982962267 (564h38m9.475394716s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.983976197 (564h38m9.476408635s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.986859791 (564h38m9.47929222s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987790541 (564h38m9.480222978s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987885955 (564h38m9.48031839s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988406341 (564h38m9.480838769s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988516453 (564h38m9.48094889s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.994795403 (564h38m9.487227836s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.995545605 (564h38m9.487978038s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.998323112 (564h38m9.490755548s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:49.003694081 (564h38m9.496126531s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:50.247858948 (564h38m10.740291397s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: connect: no route to host; conn closed: op Sep 22 02:46:17.848383509 (564h38m38.340815937s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.848632149 (564h38m38.341064593s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849472038 (564h38m38.341904483s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849621286 (564h38m38.342053718s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849657569 (564h38m38.342090015s) [seg_pos={0 0}, upload=496726]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:46:17.949774543 (564h38m38.442206977s) [upload=496726]: Done uploading segments... Sep 22 02:46:17.949816811 (564h38m38.442249241s) [upload=496726]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.360290002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...........XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.X] duration: 30.104500533s FAIL (2025-09-22 02:46:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:45:48.123977404 (564h38m8.616409847s) [upload=496726]: Starting upload Sep 22 02:45:48.124048430 (564h38m8.616480865s) [upload=496726]: Done starting upload Sep 22 02:45:48.124142375 (564h38m8.616574808s) [upload=496726]: Uploading segments... Sep 22 02:45:48.124434847 (564h38m8.61686728s) [upload=496726]: Got next segment. Inline: false Sep 22 02:45:48.124485207 (564h38m8.616917657s) [upload=496726]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:45:48.124491491 (564h38m8.616923954s) [upload=496726]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:45:48.237565267 (564h38m8.729997699s) [seg_pos={0 0}, upload=496726]: Begin upload segment... Sep 22 02:45:48.237618508 (564h38m8.730050944s) [seg_pos={0 0}, upload=496726]: Done begin upload segment. Sep 22 02:45:48.237645612 (564h38m8.730078054s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.237684929 (564h38m8.73011738s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238207044 (564h38m8.730639491s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238531941 (564h38m8.730964386s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.238846154 (564h38m8.731278605s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239244187 (564h38m8.731676641s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239755762 (564h38m8.732188207s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.239801248 (564h38m8.732233701s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240152427 (564h38m8.732584878s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240192870 (564h38m8.732625325s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240393062 (564h38m8.73282551s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240613542 (564h38m8.733045988s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240645601 (564h38m8.733078053s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.240834348 (564h38m8.733266793s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241079643 (564h38m8.733512093s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.241433198 (564h38m8.733865646s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Uploading piece... Sep 22 02:45:48.844540185 (564h38m9.33697262s) [upload=496726]: Next returned nil segment Sep 22 02:45:48.845994696 (564h38m9.338427142s) [upload=496726]: Waiting for error group managing segments... Sep 22 02:45:48.982962267 (564h38m9.475394716s) [noise=true , address=77.90.31.16:4733 , piece_id=2UYVKPQLVPF4UVCUGIQOM7VQAKK662EJV2A5WWGC6X5KECOB5MOA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.983976197 (564h38m9.476408635s) [noise=true , address=45.91.116.7:4011 , piece_id=7RIJVRWWWVQFOQML36VZCIW3LUX57RJCIKJ7ETAD6TMC3ONQQBCA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.986859791 (564h38m9.47929222s) [noise=true , address=77.90.31.19:4842 , piece_id=LXV5HZDX6NXHX3WHHNCVR2AE2ZO2APJTXAT52QG4N76TPTGVR4QA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987790541 (564h38m9.480222978s) [noise=true , address=45.91.116.13:4153 , piece_id=FJVMEUSMORHPF3ROCROSHTGCSOQS3Q33ICCFTELAGO46ELUEANYQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.987885955 (564h38m9.48031839s) [noise=true , address=77.90.31.14:4661 , piece_id=U4YWGKNNC6D5D3EYWFILG2GISCASPDPD4DUAXREXCTF45CV2CLFQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988406341 (564h38m9.480838769s) [noise=true , address=45.91.116.5:4298 , piece_id=VE5XR6RCLXU3U6VBTRPZVRV7WHUVCSUHOIYZ2BIE5OAN3Y5RV27A, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.988516453 (564h38m9.48094889s) [noise=true , address=45.91.116.4:4938 , piece_id=ZTHOC4AI6ONAPJN3YP4OHWERIHFELOMVL6SLFLSR5XSE6H4AJEAQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.994795403 (564h38m9.487227836s) [noise=true , address=77.90.31.12:4592 , piece_id=LJCAFA3NW4FVKZOM77Y5V2GM3BK2JVOFFQ2OPN7MYO7CGHEFF6XQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.995545605 (564h38m9.487978038s) [noise=true , address=45.91.116.8:4048 , piece_id=CFA5PE2H64PVWKEFMOKQ4YPPDEORXHC4CCBUEUZXFQGF53QCRQJA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:48.998323112 (564h38m9.490755548s) [noise=true , address=45.91.116.13:4154 , piece_id=YLJI7S62TICSSR7Q3UBA2Z6YVO2N2TUYLPGD6J4J6GLARHOINYAA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:49.003694081 (564h38m9.496126531s) [noise=true , address=77.90.31.18:4807 , piece_id=G6EWHIZEMAVNNLLEXVAGBPW2KPPFNY63347LNPALH3WQPHCB53KA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: <nil> Sep 22 02:45:50.247858948 (564h38m10.740291397s) [noise=true , address=145.79.227.16:4911 , piece_id=TAJBJA7OKVUA32EBWJMLCLDS7DPSGTFSZLOP2KKX6JJO6SQ3WPSQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: connect: no route to host; conn closed: op Sep 22 02:46:17.848383509 (564h38m38.340815937s) [noise=true , address=145.79.227.4:4186 , piece_id=6BLUEZPL6SYQPMML3WHGHZSD45DVWDMFOJDMVV2V5BUXSTJHD5QQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.848632149 (564h38m38.341064593s) [noise=true , address=145.79.227.11:4730 , piece_id=WTSIYFTOFDLPOZYAA3IA33BA3G5OJAZGTOKSSVKZN5VPNUCOESLQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849472038 (564h38m38.341904483s) [noise=true , address=145.79.227.13:4803 , piece_id=PAYF4ROI3RXCLPFEOZAL72PLD67QOUNYMJNJCIIQZPRYMNUECFOQ, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849621286 (564h38m38.342053718s) [noise=true , address=145.79.227.2:4113 , piece_id=BKU2KPNBV6C6OPYZPMPPHHRRDVEUGNPNMPCGVHZZ767NN5XREOMA, seg_pos={0 0}, upload=496726]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:46:17.849657569 (564h38m38.342090015s) [seg_pos={0 0}, upload=496726]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:46:17.949774543 (564h38m38.442206977s) [upload=496726]: Done uploading segments... Sep 22 02:46:17.949816811 (564h38m38.442249241s) [upload=496726]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.104533825s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................X.X.......X..X......X.X.....X.X...XX....X..] duration: 2.397003581s FAIL (2025-09-22 02:46:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.397038127s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..................X.X.......X..X......X.X.....X.X...XX....X.] duration: 2.827785762s FAIL (2025-09-22 02:45:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.827834569s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...................X.X.......X..X......X.X.....X.X...XX....X] duration: 30.001896562s FAIL (2025-09-22 02:45:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002004002s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........XXXX.X....X......................................] duration: 2.460652991s FAIL (2025-09-22 02:45:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.460696213s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....................X.X.......X..X......X.X.....X.X...XX....] duration: 2.890647375s FAIL (2025-09-22 02:44:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.890674491s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [............XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X.] duration: 2.594103793s FAIL (2025-09-22 02:44:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:44:02.972782667 (564h36m23.465215102s) [upload=496699]: Starting upload Sep 22 02:44:02.972915402 (564h36m23.465347852s) [upload=496699]: Done starting upload Sep 22 02:44:02.973127694 (564h36m23.46556014s) [upload=496699]: Uploading segments... Sep 22 02:44:02.973425557 (564h36m23.465858003s) [upload=496699]: Got next segment. Inline: false Sep 22 02:44:02.973500357 (564h36m23.465932811s) [upload=496699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:44:02.973513866 (564h36m23.465946316s) [upload=496699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:44:03.092588691 (564h36m23.58502115s) [seg_pos={0 0}, upload=496699]: Begin upload segment... Sep 22 02:44:03.092677150 (564h36m23.585109597s) [seg_pos={0 0}, upload=496699]: Done begin upload segment. Sep 22 02:44:03.092725235 (564h36m23.585157697s) [noise=true , address=145.79.227.5:4223 , piece_id=HJEJ4HYE6FQJPTJ7QESH6UAZKTKAMXPLTDSS6VMZ6CDRCDHF35EQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.092743078 (564h36m23.585175529s) [noise=true , address=45.91.116.14:4191 , piece_id=IWT32QI427QVGLN6VNATLLCRNYCEMXUH7ICHMWXQWP65AR4AB45A, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.092820050 (564h36m23.585252499s) [noise=true , address=145.79.227.13:4803 , piece_id=2XZXCLCX6RG74RPLNFM3NFR2ZOM35CSF2R7EWR2EDEYE6DR2DUIQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093246615 (564h36m23.585679065s) [noise=true , address=45.91.116.13:4155 , piece_id=IDBAFYLCUXBHEZTLYFBGJTZYWO2VS4SZTDQSOTL4UHZMSDBKOSOA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093442500 (564h36m23.585874953s) [noise=true , address=77.90.31.17:4771 , piece_id=OMRWVOVZ3RLUVM5Z3U7QQUAO3EPXGTT3DGB5QCMTNCBUEEBPBPWA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093655057 (564h36m23.586087494s) [noise=true , address=145.79.227.2:4113 , piece_id=YDHFV7WLDVTULSYBN6LLKXBHIMSVW3WCNQJNIJ422CSKUDLVHGZA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.094236060 (564h36m23.586668515s) [noise=true , address=45.91.116.10:4083 , piece_id=7JQNEWGCQLHYCU6HJUF7BVUNIJMD3442J4FCPUOEWK6RW3PJVPLA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.094771197 (564h36m23.587203648s) [noise=true , address=77.90.31.15:4699 , piece_id=QL3CWWLQADVOXOCQE742TRXEHTGLXV6MFK2DVES3IAXSTCPQPLJA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.095328937 (564h36m23.587761392s) [noise=true , address=45.91.116.8:4047 , piece_id=W6YNHC7JESPMZ5V77LDK3G5F3TI7DUX5YCYYHPVCWMHNQIOPGUUA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.095970955 (564h36m23.588403414s) [noise=true , address=145.79.227.15:4873 , piece_id=I7CQA7LZDX4EZX7D5X6ZKWX66GBJYT4WO2S3QAAQZHJSDQNYLCSA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096178450 (564h36m23.588610907s) [noise=true , address=45.91.116.5:4298 , piece_id=EE3TRFJFRB7FCP5FWQPOQMGCGBBUOK4PFGO5YW6MZKMG7JLBEIRQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096531821 (564h36m23.588964276s) [noise=true , address=77.90.31.18:4806 , piece_id=U3DJHT55Y4WOUGQSELDUFJYGPMKMJ6KZSE23UKEJK5GHDGVBD3JA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096989241 (564h36m23.589421696s) [noise=true , address=77.90.31.13:4626 , piece_id=JYHLKHTFC6G4GF6U4MQ3J34PQ4JGLPHMC4CL54FDF7VZE6EX2QPQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.097421363 (564h36m23.589853819s) [noise=true , address=45.91.116.11:4118 , piece_id=UGU6YSCTD2NCT2HUNAMFMB6DYG52MDWRYD3UK76QH6XWG27RGIGQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.097832140 (564h36m23.590264595s) [noise=true , address=45.91.116.13:4153 , piece_id=NZLGZDPIDZU4FZLUYCMGW3GUY4SAUEFN3HMAYK6P4T64I6CISHYA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.098292482 (564h36m23.590724938s) [noise=true , address=45.91.116.12:4226 , piece_id=O7QMQ5NHRWCB6J2444DHPK3727U46TGL7ZZ72ZMXUZ77PXPLPFXA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.738541907 (564h36m24.230974346s) [upload=496699]: Next returned nil segment Sep 22 02:44:03.738592413 (564h36m24.231024859s) [upload=496699]: Waiting for error group managing segments... Sep 22 02:44:03.880665160 (564h36m24.373097608s) [noise=true , address=45.91.116.11:4118 , piece_id=UGU6YSCTD2NCT2HUNAMFMB6DYG52MDWRYD3UK76QH6XWG27RGIGQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.882525035 (564h36m24.374957493s) [noise=true , address=45.91.116.13:4155 , piece_id=IDBAFYLCUXBHEZTLYFBGJTZYWO2VS4SZTDQSOTL4UHZMSDBKOSOA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.883997910 (564h36m24.376430374s) [noise=true , address=77.90.31.17:4771 , piece_id=OMRWVOVZ3RLUVM5Z3U7QQUAO3EPXGTT3DGB5QCMTNCBUEEBPBPWA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.884611419 (564h36m24.377043865s) [noise=true , address=45.91.116.5:4298 , piece_id=EE3TRFJFRB7FCP5FWQPOQMGCGBBUOK4PFGO5YW6MZKMG7JLBEIRQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.885216500 (564h36m24.377648946s) [noise=true , address=45.91.116.12:4226 , piece_id=O7QMQ5NHRWCB6J2444DHPK3727U46TGL7ZZ72ZMXUZ77PXPLPFXA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.885831904 (564h36m24.378264346s) [noise=true , address=77.90.31.18:4806 , piece_id=U3DJHT55Y4WOUGQSELDUFJYGPMKMJ6KZSE23UKEJK5GHDGVBD3JA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.886426130 (564h36m24.378858559s) [noise=true , address=45.91.116.10:4083 , piece_id=7JQNEWGCQLHYCU6HJUF7BVUNIJMD3442J4FCPUOEWK6RW3PJVPLA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.887029537 (564h36m24.379461983s) [noise=true , address=45.91.116.14:4191 , piece_id=IWT32QI427QVGLN6VNATLLCRNYCEMXUH7ICHMWXQWP65AR4AB45A, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.887684731 (564h36m24.380117175s) [noise=true , address=77.90.31.13:4626 , piece_id=JYHLKHTFC6G4GF6U4MQ3J34PQ4JGLPHMC4CL54FDF7VZE6EX2QPQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.888480063 (564h36m24.380912516s) [noise=true , address=45.91.116.8:4047 , piece_id=W6YNHC7JESPMZ5V77LDK3G5F3TI7DUX5YCYYHPVCWMHNQIOPGUUA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.889062262 (564h36m24.381494693s) [noise=true , address=45.91.116.13:4153 , piece_id=NZLGZDPIDZU4FZLUYCMGW3GUY4SAUEFN3HMAYK6P4T64I6CISHYA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.905318934 (564h36m24.397751362s) [noise=true , address=77.90.31.15:4699 , piece_id=QL3CWWLQADVOXOCQE742TRXEHTGLXV6MFK2DVES3IAXSTCPQPLJA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:32.671967096 (564h36m53.16439953s) [noise=true , address=145.79.227.5:4223 , piece_id=HJEJ4HYE6FQJPTJ7QESH6UAZKTKAMXPLTDSS6VMZ6CDRCDHF35EQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672033168 (564h36m53.164465604s) [noise=true , address=145.79.227.13:4803 , piece_id=2XZXCLCX6RG74RPLNFM3NFR2ZOM35CSF2R7EWR2EDEYE6DR2DUIQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672059640 (564h36m53.164492074s) [noise=true , address=145.79.227.2:4113 , piece_id=YDHFV7WLDVTULSYBN6LLKXBHIMSVW3WCNQJNIJ422CSKUDLVHGZA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672107889 (564h36m53.164540333s) [noise=true , address=145.79.227.15:4873 , piece_id=I7CQA7LZDX4EZX7D5X6ZKWX66GBJYT4WO2S3QAAQZHJSDQNYLCSA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672167253 (564h36m53.164599697s) [seg_pos={0 0}, upload=496699]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:44:32.782574445 (564h36m53.275006881s) [upload=496699]: Done uploading segments... Sep 22 02:44:32.782613016 (564h36m53.275045461s) [upload=496699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.59413111s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [.............XXXXXXX..X.X...XXXXXXX.X.X.X.XX.........X..X..X] duration: 30.113177985s FAIL (2025-09-22 02:44:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:44:02.972782667 (564h36m23.465215102s) [upload=496699]: Starting upload Sep 22 02:44:02.972915402 (564h36m23.465347852s) [upload=496699]: Done starting upload Sep 22 02:44:02.973127694 (564h36m23.46556014s) [upload=496699]: Uploading segments... Sep 22 02:44:02.973425557 (564h36m23.465858003s) [upload=496699]: Got next segment. Inline: false Sep 22 02:44:02.973500357 (564h36m23.465932811s) [upload=496699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:44:02.973513866 (564h36m23.465946316s) [upload=496699]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:44:03.092588691 (564h36m23.58502115s) [seg_pos={0 0}, upload=496699]: Begin upload segment... Sep 22 02:44:03.092677150 (564h36m23.585109597s) [seg_pos={0 0}, upload=496699]: Done begin upload segment. Sep 22 02:44:03.092725235 (564h36m23.585157697s) [noise=true , address=145.79.227.5:4223 , piece_id=HJEJ4HYE6FQJPTJ7QESH6UAZKTKAMXPLTDSS6VMZ6CDRCDHF35EQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.092743078 (564h36m23.585175529s) [noise=true , address=45.91.116.14:4191 , piece_id=IWT32QI427QVGLN6VNATLLCRNYCEMXUH7ICHMWXQWP65AR4AB45A, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.092820050 (564h36m23.585252499s) [noise=true , address=145.79.227.13:4803 , piece_id=2XZXCLCX6RG74RPLNFM3NFR2ZOM35CSF2R7EWR2EDEYE6DR2DUIQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093246615 (564h36m23.585679065s) [noise=true , address=45.91.116.13:4155 , piece_id=IDBAFYLCUXBHEZTLYFBGJTZYWO2VS4SZTDQSOTL4UHZMSDBKOSOA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093442500 (564h36m23.585874953s) [noise=true , address=77.90.31.17:4771 , piece_id=OMRWVOVZ3RLUVM5Z3U7QQUAO3EPXGTT3DGB5QCMTNCBUEEBPBPWA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.093655057 (564h36m23.586087494s) [noise=true , address=145.79.227.2:4113 , piece_id=YDHFV7WLDVTULSYBN6LLKXBHIMSVW3WCNQJNIJ422CSKUDLVHGZA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.094236060 (564h36m23.586668515s) [noise=true , address=45.91.116.10:4083 , piece_id=7JQNEWGCQLHYCU6HJUF7BVUNIJMD3442J4FCPUOEWK6RW3PJVPLA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.094771197 (564h36m23.587203648s) [noise=true , address=77.90.31.15:4699 , piece_id=QL3CWWLQADVOXOCQE742TRXEHTGLXV6MFK2DVES3IAXSTCPQPLJA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.095328937 (564h36m23.587761392s) [noise=true , address=45.91.116.8:4047 , piece_id=W6YNHC7JESPMZ5V77LDK3G5F3TI7DUX5YCYYHPVCWMHNQIOPGUUA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.095970955 (564h36m23.588403414s) [noise=true , address=145.79.227.15:4873 , piece_id=I7CQA7LZDX4EZX7D5X6ZKWX66GBJYT4WO2S3QAAQZHJSDQNYLCSA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096178450 (564h36m23.588610907s) [noise=true , address=45.91.116.5:4298 , piece_id=EE3TRFJFRB7FCP5FWQPOQMGCGBBUOK4PFGO5YW6MZKMG7JLBEIRQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096531821 (564h36m23.588964276s) [noise=true , address=77.90.31.18:4806 , piece_id=U3DJHT55Y4WOUGQSELDUFJYGPMKMJ6KZSE23UKEJK5GHDGVBD3JA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.096989241 (564h36m23.589421696s) [noise=true , address=77.90.31.13:4626 , piece_id=JYHLKHTFC6G4GF6U4MQ3J34PQ4JGLPHMC4CL54FDF7VZE6EX2QPQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.097421363 (564h36m23.589853819s) [noise=true , address=45.91.116.11:4118 , piece_id=UGU6YSCTD2NCT2HUNAMFMB6DYG52MDWRYD3UK76QH6XWG27RGIGQ, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.097832140 (564h36m23.590264595s) [noise=true , address=45.91.116.13:4153 , piece_id=NZLGZDPIDZU4FZLUYCMGW3GUY4SAUEFN3HMAYK6P4T64I6CISHYA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.098292482 (564h36m23.590724938s) [noise=true , address=45.91.116.12:4226 , piece_id=O7QMQ5NHRWCB6J2444DHPK3727U46TGL7ZZ72ZMXUZ77PXPLPFXA, seg_pos={0 0}, upload=496699]: Uploading piece... Sep 22 02:44:03.738541907 (564h36m24.230974346s) [upload=496699]: Next returned nil segment Sep 22 02:44:03.738592413 (564h36m24.231024859s) [upload=496699]: Waiting for error group managing segments... Sep 22 02:44:03.880665160 (564h36m24.373097608s) [noise=true , address=45.91.116.11:4118 , piece_id=UGU6YSCTD2NCT2HUNAMFMB6DYG52MDWRYD3UK76QH6XWG27RGIGQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.882525035 (564h36m24.374957493s) [noise=true , address=45.91.116.13:4155 , piece_id=IDBAFYLCUXBHEZTLYFBGJTZYWO2VS4SZTDQSOTL4UHZMSDBKOSOA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.883997910 (564h36m24.376430374s) [noise=true , address=77.90.31.17:4771 , piece_id=OMRWVOVZ3RLUVM5Z3U7QQUAO3EPXGTT3DGB5QCMTNCBUEEBPBPWA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.884611419 (564h36m24.377043865s) [noise=true , address=45.91.116.5:4298 , piece_id=EE3TRFJFRB7FCP5FWQPOQMGCGBBUOK4PFGO5YW6MZKMG7JLBEIRQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.885216500 (564h36m24.377648946s) [noise=true , address=45.91.116.12:4226 , piece_id=O7QMQ5NHRWCB6J2444DHPK3727U46TGL7ZZ72ZMXUZ77PXPLPFXA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.885831904 (564h36m24.378264346s) [noise=true , address=77.90.31.18:4806 , piece_id=U3DJHT55Y4WOUGQSELDUFJYGPMKMJ6KZSE23UKEJK5GHDGVBD3JA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.886426130 (564h36m24.378858559s) [noise=true , address=45.91.116.10:4083 , piece_id=7JQNEWGCQLHYCU6HJUF7BVUNIJMD3442J4FCPUOEWK6RW3PJVPLA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.887029537 (564h36m24.379461983s) [noise=true , address=45.91.116.14:4191 , piece_id=IWT32QI427QVGLN6VNATLLCRNYCEMXUH7ICHMWXQWP65AR4AB45A, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.887684731 (564h36m24.380117175s) [noise=true , address=77.90.31.13:4626 , piece_id=JYHLKHTFC6G4GF6U4MQ3J34PQ4JGLPHMC4CL54FDF7VZE6EX2QPQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.888480063 (564h36m24.380912516s) [noise=true , address=45.91.116.8:4047 , piece_id=W6YNHC7JESPMZ5V77LDK3G5F3TI7DUX5YCYYHPVCWMHNQIOPGUUA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.889062262 (564h36m24.381494693s) [noise=true , address=45.91.116.13:4153 , piece_id=NZLGZDPIDZU4FZLUYCMGW3GUY4SAUEFN3HMAYK6P4T64I6CISHYA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:03.905318934 (564h36m24.397751362s) [noise=true , address=77.90.31.15:4699 , piece_id=QL3CWWLQADVOXOCQE742TRXEHTGLXV6MFK2DVES3IAXSTCPQPLJA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: <nil> Sep 22 02:44:32.671967096 (564h36m53.16439953s) [noise=true , address=145.79.227.5:4223 , piece_id=HJEJ4HYE6FQJPTJ7QESH6UAZKTKAMXPLTDSS6VMZ6CDRCDHF35EQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672033168 (564h36m53.164465604s) [noise=true , address=145.79.227.13:4803 , piece_id=2XZXCLCX6RG74RPLNFM3NFR2ZOM35CSF2R7EWR2EDEYE6DR2DUIQ, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: rpc: dial tcp 145.79.227.13:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672059640 (564h36m53.164492074s) [noise=true , address=145.79.227.2:4113 , piece_id=YDHFV7WLDVTULSYBN6LLKXBHIMSVW3WCNQJNIJ422CSKUDLVHGZA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672107889 (564h36m53.164540333s) [noise=true , address=145.79.227.15:4873 , piece_id=I7CQA7LZDX4EZX7D5X6ZKWX66GBJYT4WO2S3QAAQZHJSDQNYLCSA, seg_pos={0 0}, upload=496699]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:44:32.672167253 (564h36m53.164599697s) [seg_pos={0 0}, upload=496699]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:44:32.782574445 (564h36m53.275006881s) [upload=496699]: Done uploading segments... Sep 22 02:44:32.782613016 (564h36m53.275045461s) [upload=496699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.113204051s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....................X.X.......X..X......X.X.....X.X...XX...] duration: 2.899757633s FAIL (2025-09-22 02:44:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.899797982s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [............XXXX.X....X.....................................] duration: 2.581949747s FAIL (2025-09-22 02:44:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.581988901s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............XXXX.X....X....................................] duration: 2.713820289s FAIL (2025-09-22 02:44:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.713912783s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............XXXX.X....X...................................] duration: 2.74542276s FAIL (2025-09-22 02:43:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.745460077s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......................X.X.......X..X......X.X.....X.X...XX..] duration: 2.803221596s FAIL (2025-09-22 02:43:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.80324534s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)