Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 173220 reports (Page 374 of 3465)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XXXX.X....X...] duration: 2.682759339s FAIL (2025-09-22 02:18:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.682802902s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................................XXXXXXX..X.X] duration: 30.112588839s FAIL (2025-09-22 02:17:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:17:27.278111860 (564h9m47.770544312s) [upload=496310]: Starting upload Sep 22 02:17:27.278319587 (564h9m47.770752032s) [upload=496310]: Done starting upload Sep 22 02:17:27.278399633 (564h9m47.770832069s) [upload=496310]: Uploading segments... Sep 22 02:17:27.278679186 (564h9m47.771111629s) [upload=496310]: Got next segment. Inline: false Sep 22 02:17:27.278716987 (564h9m47.771149429s) [upload=496310]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:17:27.278723741 (564h9m47.771156183s) [upload=496310]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:17:27.391606475 (564h9m47.88403892s) [seg_pos={0 0}, upload=496310]: Begin upload segment... Sep 22 02:17:27.391656507 (564h9m47.884088952s) [seg_pos={0 0}, upload=496310]: Done begin upload segment. Sep 22 02:17:27.391684205 (564h9m47.884116647s) [noise=true , address=145.79.227.18:4006 , piece_id=LG4XNGKQYGDXAAT3N5326M2LZNJIF4567R3YDVRVTUEEZ3AXGPAQ, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.391694105 (564h9m47.884126551s) [noise=true , address=45.91.116.6:4973 , piece_id=Q7XJW3INL5BOB2GML6IXP7UVBSUH5B5ZPYJOMRNA2SJCT322QWTA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.392068435 (564h9m47.884500876s) [noise=true , address=145.79.227.14:4840 , piece_id=7OV75GGOIRGJJ3UJ6KA2625HLQM5O7FYYIVKXZCKWWOBJXKP6J2A, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.392079800 (564h9m47.884512242s) [noise=true , address=45.91.116.14:4191 , piece_id=BWEGJXAOKTPUIUZTPYIAJOF6IZASOSLHDVCYO4E475HB7RB2JQXQ, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.392359813 (564h9m47.884792255s) [noise=true , address=145.79.227.4:4186 , piece_id=X2S2O7G2TO7W4NNTCHOUGTACSVAYPSJH4NNFXVNNXEUXQJSTGB6A, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.392866275 (564h9m47.885298718s) [noise=true , address=45.91.116.13:4155 , piece_id=OKVAXFDLQDEXO4UBC4RFX44FIIKDHM6OSEL5QOV3ISQAOUJ5RSFA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.393199150 (564h9m47.885631585s) [noise=true , address=45.91.116.9:4261 , piece_id=BTQ6KKVFJWU732FB4YFSNO6HEINBXWAHVSU6FXPQHMSCXBWATZHA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.394785094 (564h9m47.887217537s) [noise=true , address=77.90.31.16:4734 , piece_id=24UDK3VHFEE242TEPT4OHFU26TW3PQ7C4NY5WGYRTDV7T37A22EA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.395727720 (564h9m47.888160168s) [noise=true , address=77.90.31.19:4843 , piece_id=O565PVZICSRTDI7DG3XVUM5REFUOXNEJT5G23UFFKWTQ5HBGRSLA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.396064317 (564h9m47.88849676s) [noise=true , address=145.79.227.15:4876 , piece_id=CATK7VYS34V6FM2TDA6WP6ZSCVL42WCRJQGMKMQE6B3KL2RUX5GA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.397050510 (564h9m47.889482958s) [noise=true , address=145.79.227.19:4079 , piece_id=GOC7R5364ANS74BUMZIKFYOQE5IOF4WIYYMBSQO7L5LFKH7T6MLA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.398371111 (564h9m47.890803571s) [noise=true , address=145.79.227.16:4912 , piece_id=WAWYX2DPWKMT2DWUQPLE3B5QR277HOYQJKAJ5RIPNQGY62V3FU2A, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.404113314 (564h9m47.896545774s) [noise=true , address=45.91.116.4:4937 , piece_id=PTEXK44Q5HARZBHVNXMIJQUYTYOEVR6JG7KOJTJTHX27AVO3V3IA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.404596127 (564h9m47.897028571s) [noise=true , address=145.79.227.16:4911 , piece_id=JCHWX24SAGWM67MQXDFY42FK7YXVREHIM4JBIXBKDZCQDHAPUTOA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.404928472 (564h9m47.897360901s) [noise=true , address=45.91.116.7:4010 , piece_id=SVNDPUPAZFAZH6ECKRTKZHTWO2CWBDFMC3ZHVA7YATN7EGAJYQYA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.411543951 (564h9m47.90397638s) [noise=true , address=45.91.116.13:4153 , piece_id=ZHHJ7HWJ3BZ7SOELVG5PKV7BECDBC6OTUPPWMX2IOKWOBFY5RIZA, seg_pos={0 0}, upload=496310]: Uploading piece... Sep 22 02:17:27.994508946 (564h9m48.486941392s) [upload=496310]: Next returned nil segment Sep 22 02:17:27.994529653 (564h9m48.4869621s) [upload=496310]: Waiting for error group managing segments... Sep 22 02:17:28.133038661 (564h9m48.625471091s) [noise=true , address=45.91.116.9:4261 , piece_id=BTQ6KKVFJWU732FB4YFSNO6HEINBXWAHVSU6FXPQHMSCXBWATZHA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.134870648 (564h9m48.627303084s) [noise=true , address=45.91.116.6:4973 , piece_id=Q7XJW3INL5BOB2GML6IXP7UVBSUH5B5ZPYJOMRNA2SJCT322QWTA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.136919100 (564h9m48.629351545s) [noise=true , address=45.91.116.13:4155 , piece_id=OKVAXFDLQDEXO4UBC4RFX44FIIKDHM6OSEL5QOV3ISQAOUJ5RSFA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.159786394 (564h9m48.65221883s) [noise=true , address=77.90.31.19:4843 , piece_id=O565PVZICSRTDI7DG3XVUM5REFUOXNEJT5G23UFFKWTQ5HBGRSLA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.161298102 (564h9m48.653730546s) [noise=true , address=45.91.116.4:4937 , piece_id=PTEXK44Q5HARZBHVNXMIJQUYTYOEVR6JG7KOJTJTHX27AVO3V3IA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.165445218 (564h9m48.65787766s) [noise=true , address=45.91.116.13:4153 , piece_id=ZHHJ7HWJ3BZ7SOELVG5PKV7BECDBC6OTUPPWMX2IOKWOBFY5RIZA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.170034775 (564h9m48.662467211s) [noise=true , address=77.90.31.16:4734 , piece_id=24UDK3VHFEE242TEPT4OHFU26TW3PQ7C4NY5WGYRTDV7T37A22EA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.173136107 (564h9m48.665568551s) [noise=true , address=45.91.116.7:4010 , piece_id=SVNDPUPAZFAZH6ECKRTKZHTWO2CWBDFMC3ZHVA7YATN7EGAJYQYA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:28.176492195 (564h9m48.668924642s) [noise=true , address=45.91.116.14:4191 , piece_id=BWEGJXAOKTPUIUZTPYIAJOF6IZASOSLHDVCYO4E475HB7RB2JQXQ, seg_pos={0 0}, upload=496310]: Done uploading piece. err: <nil> Sep 22 02:17:56.985562546 (564h10m17.477994998s) [noise=true , address=145.79.227.19:4079 , piece_id=GOC7R5364ANS74BUMZIKFYOQE5IOF4WIYYMBSQO7L5LFKH7T6MLA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:17:56.985662732 (564h10m17.478095164s) [noise=true , address=145.79.227.16:4912 , piece_id=WAWYX2DPWKMT2DWUQPLE3B5QR277HOYQJKAJ5RIPNQGY62V3FU2A, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.985716627 (564h10m17.478149059s) [noise=true , address=145.79.227.4:4186 , piece_id=X2S2O7G2TO7W4NNTCHOUGTACSVAYPSJH4NNFXVNNXEUXQJSTGB6A, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.985769530 (564h10m17.478201972s) [noise=true , address=145.79.227.15:4876 , piece_id=CATK7VYS34V6FM2TDA6WP6ZSCVL42WCRJQGMKMQE6B3KL2RUX5GA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.985812353 (564h10m17.478244796s) [noise=true , address=145.79.227.14:4840 , piece_id=7OV75GGOIRGJJ3UJ6KA2625HLQM5O7FYYIVKXZCKWWOBJXKP6J2A, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.985884301 (564h10m17.478316742s) [noise=true , address=145.79.227.16:4911 , piece_id=JCHWX24SAGWM67MQXDFY42FK7YXVREHIM4JBIXBKDZCQDHAPUTOA, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: rpc: dial tcp 145.79.227.16:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.985979172 (564h10m17.478411614s) [noise=true , address=145.79.227.18:4006 , piece_id=LG4XNGKQYGDXAAT3N5326M2LZNJIF4567R3YDVRVTUEEZ3AXGPAQ, seg_pos={0 0}, upload=496310]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: rpc: dial tcp 145.79.227.18:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:56.986000804 (564h10m17.478433247s) [seg_pos={0 0}, upload=496310]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:17:57.096130906 (564h10m17.588563349s) [upload=496310]: Done uploading segments... Sep 22 02:17:57.096161057 (564h10m17.5885935s) [upload=496310]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.112636189s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXXX.X....X..] duration: 2.665506586s FAIL (2025-09-22 02:17:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.665538954s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.879501095s FAIL (2025-09-22 02:17:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.879519121s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXXX.X....X.] duration: 2.630030464s FAIL (2025-09-22 02:17:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.630096782s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXXX.X....X] duration: 1m0.156731006s FAIL (2025-09-22 02:17:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== 0159s) [seg_pos={0 0}, upload=496292]: Begin upload segment... Sep 22 02:16:16.443075073 (564h8m36.935507503s) [seg_pos={0 0}, upload=496292]: Done begin upload segment. Sep 22 02:16:16.443105441 (564h8m36.935537893s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.443347197 (564h8m36.935779648s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444152041 (564h8m36.93658457s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444600279 (564h8m36.937032731s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.444883603 (564h8m36.937316047s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445112196 (564h8m36.937544647s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.445956555 (564h8m36.938389007s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.446278645 (564h8m36.938711093s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447121359 (564h8m36.939553811s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447247470 (564h8m36.939679921s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.447700559 (564h8m36.940133007s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448088869 (564h8m36.940521323s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448540353 (564h8m36.940972802s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448647427 (564h8m36.941079875s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448905693 (564h8m36.941338139s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.448985663 (564h8m36.941418114s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.449364887 (564h8m36.941797333s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:16.450441809 (564h8m36.942874258s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Uploading piece... Sep 22 02:16:17.016630581 (564h8m37.509063022s) [upload=496292]: Next returned nil segment Sep 22 02:16:17.016651972 (564h8m37.509084415s) [upload=496292]: Waiting for error group managing segments... Sep 22 02:16:17.163540560 (564h8m37.655973015s) [noise=true , address=77.90.31.19:4871 , piece_id=TB3Z4XV77ZKBEG3E4MJMUZIJ6XKJM4ILIBARRFU7XIUV2KO4UGPQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.171809778 (564h8m37.664242211s) [noise=true , address=45.91.116.12:4232 , piece_id=WIKBPCGQXX7FGG4N3FJT36FPQ5TVJS5UP43J7K77WZSQIFMTZLVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.195795139 (564h8m37.688227574s) [noise=true , address=145.79.227.31:4831 , piece_id=MVSUPUPNT7EI4PMXWGNZ75ROQ62C5NQLIJONWKRBSYMFBR5XVVUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.198518034 (564h8m37.690950469s) [noise=true , address=145.79.227.21:4500 , piece_id=KBLWTMLGDKXN2HDK3PKXMVDIFSOCG6JORCH7TW7NWMYZPGSTUCKA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.205267668 (564h8m37.697700103s) [noise=true , address=145.79.227.26:4680 , piece_id=FQELAKU4S5ISTJD3CQG6TXWLS5G7KB42ED7XHFQTMEE5QMWYVSTQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.212402984 (564h8m37.704835418s) [noise=true , address=145.79.227.45:4260 , piece_id=EOCQ2LDBF2TB63PQEK5QIVIUEY6XJIWJ63Y5D3IFGKWQJIC7VAVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.216238221 (564h8m37.708670657s) [noise=true , address=145.79.227.52:4467 , piece_id=VPYBXZVAYVYYIW55NCDDZW5JIB4UUDBGW26YPDE4P6OEKOL3RVMQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.218275922 (564h8m37.710708358s) [noise=true , address=145.79.227.33:4900 , piece_id=2FDOCBNM3CRDKCKMWQPP6U6FBWQ7FUNS7CTITSH52L5EWQ4WENVQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.222720118 (564h8m37.715152552s) [noise=true , address=145.79.227.24:4591 , piece_id=UGHKZVNIPSTIMSX7SOXU2WBZMYFGAN7P524KGNR2B6GQXYMREFOA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.234576123 (564h8m37.727008552s) [noise=true , address=145.79.227.20:4450 , piece_id=C5F4WXTRZOBDRQM42HYC3TCDI2EB3KX625UBO57VYJJDKHHUIWGA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:16:17.269823246 (564h8m37.762255674s) [noise=true , address=145.79.227.27:4720 , piece_id=JPZ5PCHIKPVLDQOJ3UKKWLBY2KDT2EF6EZSSZUBULXELMDECABEA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: <nil> Sep 22 02:17:15.918034470 (564h9m36.410466903s) [noise=true , address=145.79.227.12:4787 , piece_id=IBVLLQFXHZZSML7RE5O6MRC44GBMPRUGOOXIW5VSW3DJUEIJOENA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1HiqDGVv9cDPmxCNevHz7HYV6Sna6KqHhgeBkys6zcrKZMXSyh, address:145.79.227.12:4787): protocol: rpc: dial tcp 145.79.227.12:4787: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918254316 (564h9m36.410686753s) [noise=true , address=145.79.227.13:4836 , piece_id=2U4QXKGXUXPYCIYVI5BAMKQSYVP2REYVW2UMAZEUYZORI72C433A, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:1fjbyyAnz2pYU8drcBLyd1VzHgsNSWAxdWXovYJWzTcSdANzNe, address:145.79.227.13:4836): protocol: rpc: dial tcp 145.79.227.13:4836: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918319211 (564h9m36.410751654s) [noise=true , address=145.79.227.16:4923 , piece_id=AXJ2HPZOGJHXJF52GB2LOX3EFQGSXEFJHK2YC66BKDHTDSZNBFUA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: rpc: dial tcp 145.79.227.16:4923: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918450129 (564h9m36.410882559s) [noise=true , address=145.79.227.5:4227 , piece_id=FOFYSH67V3LQIIBAEBGYDNKPNYPTPZQ6P2VB6XUBWAOTT3VJZKAQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: rpc: dial tcp 145.79.227.5:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918489281 (564h9m36.41092171s) [noise=true , address=145.79.227.16:4924 , piece_id=VXKOIJMDQ5YHZCTNG46VGRYEPWS5D7BI7GZ24UGQRHKJUSZGDZQQ, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12mXBUcQEzsNgQ1YQzU9ezpZigBEHHhPttBac2G1Xrjks3SwM6z, address:145.79.227.16:4924): protocol: rpc: dial tcp 145.79.227.16:4924: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918527082 (564h9m36.410959511s) [noise=true , address=145.79.227.11:4753 , piece_id=LT3SZC6Z7P6MKGK7H6W5KAAOSM4V2IMPCAOLMM4XRKAIIUAVEFZA, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12RvrzrVY3N4FD2iNFBkHYBgitmrfcrLreLN6mJARGobdpbgtGu, address:145.79.227.11:4753): protocol: rpc: dial tcp 145.79.227.11:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918538378 (564h9m36.410970823s) [noise=true , address=145.79.227.7:4324 , piece_id=TYC5E2IKQVW4AA5M5I34RNCZ5R6IHFEPALOZKHV2MJV3JZ3V2O2Q, seg_pos={0 0}, upload=496292]: Done uploading piece. err: ecclient: upload failed (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga, address:145.79.227.7:4324): protocol: rpc: dial tcp 145.79.227.7:4324: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:17:15.918575562 (564h9m36.411008016s) [seg_pos={0 0}, upload=496292]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:17:16.071853105 (564h9m36.564285546s) [upload=496292]: Done uploading segments... Sep 22 02:17:16.071889485 (564h9m36.564321933s) [upload=496292]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.156752701s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001271496s FAIL (2025-09-22 02:17:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001298921s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................................XXXXXXX..X.] duration: 2.408722761s FAIL (2025-09-22 02:16:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:15:43.060999338 (564h8m3.55343178s) [upload=496286]: Starting upload Sep 22 02:15:43.061082211 (564h8m3.553514654s) [upload=496286]: Done starting upload Sep 22 02:15:43.061220388 (564h8m3.553652823s) [upload=496286]: Uploading segments... Sep 22 02:15:43.061425366 (564h8m3.5538578s) [upload=496286]: Got next segment. Inline: false Sep 22 02:15:43.061501012 (564h8m3.553933464s) [upload=496286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:15:43.061527067 (564h8m3.553959517s) [upload=496286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:15:43.174021732 (564h8m3.666454178s) [seg_pos={0 0}, upload=496286]: Begin upload segment... Sep 22 02:15:43.174133648 (564h8m3.666566091s) [seg_pos={0 0}, upload=496286]: Done begin upload segment. Sep 22 02:15:43.174162119 (564h8m3.666594563s) [noise=true , address=145.79.227.6:4258 , piece_id=LDCDWV447WOJYW3LPMOECURNXHOBQBS6GZXXA2ETGS6QGI62SPDA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174220457 (564h8m3.666652906s) [noise=true , address=145.79.227.11:4730 , piece_id=JHKHU43WPHKIKYPMJJR3HVM73B6KL2GEBDYBMNKJM5BFAJKK4OQQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174581055 (564h8m3.667013523s) [noise=true , address=45.91.116.4:4939 , piece_id=AJAIWWFDDCHLFRQZ3KIC6UGIZBPYXAERO2AHPUTSLREFLDJVDA7Q, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174728551 (564h8m3.667160996s) [noise=true , address=45.91.116.13:4155 , piece_id=TUAILL2Z76LYEV7JDEXCMGM62EHAOSYYPZJVJ3XMEAN7FC7X54WQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174882432 (564h8m3.667314878s) [noise=true , address=145.79.227.7:4293 , piece_id=REYRRE5XZNS2RHEJCUU77U335VSQYX7NN4EPS5WTJGITP7RF54JQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.175172953 (564h8m3.667605626s) [noise=true , address=145.79.227.2:4113 , piece_id=RCMMPOCO42UV5U3FZKGANAT26N7OCM3CJZHV46HLKGHMCWH7376Q, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.175517872 (564h8m3.667950317s) [noise=true , address=45.91.116.9:4262 , piece_id=T25DH4GQGBLRBO2CM2KEU23IQJOZQHFUO26ROJUH6VB5TMZ3RZJA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.177522891 (564h8m3.669955342s) [noise=true , address=45.91.116.12:4226 , piece_id=7WIQD2XI3C6NOE6NKHKRDHROHDZLHZERCNS3QH3CG6ZIUFZAZ3KA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.177857355 (564h8m3.670289809s) [noise=true , address=45.91.116.5:4299 , piece_id=DI2YSURBCMM6EVGRPOI7B5BBIUVH3A3TNUVK7CC2CJ5L6OWXNDHQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.178483650 (564h8m3.670916098s) [noise=true , address=145.79.227.13:4804 , piece_id=WPGKDLRIVO5TFWTTZU6TJOF6HEEXSR3C6J2K377H3WWSZC5RO3IQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.178817920 (564h8m3.671250379s) [noise=true , address=145.79.227.14:4838 , piece_id=PZDXWEBW2X6ZYP3HWI6BQL5YY2U7TXFUEB52AVMWIKL2KV3SX43A, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179109436 (564h8m3.671541879s) [noise=true , address=45.91.116.6:4973 , piece_id=AKCKIXSHRTNU6N55QWWB5EW5TS6QB6SELPP4WSLN65KFKYMYU3MA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179392435 (564h8m3.671824881s) [noise=true , address=45.91.116.13:4154 , piece_id=IJOYTF2URZOSWDFYN6KO3P7LITBYU6QIESWFEASHVUGUECMJJEDA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179666709 (564h8m3.672099151s) [noise=true , address=77.90.31.19:4843 , piece_id=J3K3HIPB7N2RUJ5SIJRCRPPWHIVVO62M5UGQD4LRGMBRZADVEOFA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179956688 (564h8m3.672389136s) [noise=true , address=45.91.116.8:4048 , piece_id=2EUC7VPHEQBDKZZLZ622DCJEMSU2CV22ZKHVIMCZAY6BL7Z4B3MA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.180280762 (564h8m3.672713215s) [noise=true , address=145.79.227.16:4912 , piece_id=VQT36O7YTR2V3TR7AUZXM5KTV6JLGJRG7NDZFDHIGQX6ZVWLLQOQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.792732778 (564h8m4.285165211s) [upload=496286]: Next returned nil segment Sep 22 02:15:43.792762497 (564h8m4.285194943s) [upload=496286]: Waiting for error group managing segments... Sep 22 02:15:43.928148341 (564h8m4.420580769s) [noise=true , address=45.91.116.4:4939 , piece_id=AJAIWWFDDCHLFRQZ3KIC6UGIZBPYXAERO2AHPUTSLREFLDJVDA7Q, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.932360246 (564h8m4.42479269s) [noise=true , address=45.91.116.6:4973 , piece_id=AKCKIXSHRTNU6N55QWWB5EW5TS6QB6SELPP4WSLN65KFKYMYU3MA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.934574498 (564h8m4.427006931s) [noise=true , address=45.91.116.13:4155 , piece_id=TUAILL2Z76LYEV7JDEXCMGM62EHAOSYYPZJVJ3XMEAN7FC7X54WQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.936508997 (564h8m4.42894144s) [noise=true , address=45.91.116.9:4262 , piece_id=T25DH4GQGBLRBO2CM2KEU23IQJOZQHFUO26ROJUH6VB5TMZ3RZJA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.940156209 (564h8m4.432588656s) [noise=true , address=77.90.31.19:4843 , piece_id=J3K3HIPB7N2RUJ5SIJRCRPPWHIVVO62M5UGQD4LRGMBRZADVEOFA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.943311504 (564h8m4.435743948s) [noise=true , address=45.91.116.5:4299 , piece_id=DI2YSURBCMM6EVGRPOI7B5BBIUVH3A3TNUVK7CC2CJ5L6OWXNDHQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.946116925 (564h8m4.438549372s) [noise=true , address=45.91.116.13:4154 , piece_id=IJOYTF2URZOSWDFYN6KO3P7LITBYU6QIESWFEASHVUGUECMJJEDA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.947998253 (564h8m4.440430697s) [noise=true , address=45.91.116.8:4048 , piece_id=2EUC7VPHEQBDKZZLZ622DCJEMSU2CV22ZKHVIMCZAY6BL7Z4B3MA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.957203589 (564h8m4.449636027s) [noise=true , address=45.91.116.12:4226 , piece_id=7WIQD2XI3C6NOE6NKHKRDHROHDZLHZERCNS3QH3CG6ZIUFZAZ3KA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:16:12.770990645 (564h8m33.263423088s) [noise=true , address=145.79.227.14:4838 , piece_id=PZDXWEBW2X6ZYP3HWI6BQL5YY2U7TXFUEB52AVMWIKL2KV3SX43A, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771183282 (564h8m33.263615712s) [noise=true , address=145.79.227.6:4258 , piece_id=LDCDWV447WOJYW3LPMOECURNXHOBQBS6GZXXA2ETGS6QGI62SPDA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771269437 (564h8m33.263701879s) [noise=true , address=145.79.227.7:4293 , piece_id=REYRRE5XZNS2RHEJCUU77U335VSQYX7NN4EPS5WTJGITP7RF54JQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771339130 (564h8m33.263771573s) [noise=true , address=145.79.227.16:4912 , piece_id=VQT36O7YTR2V3TR7AUZXM5KTV6JLGJRG7NDZFDHIGQX6ZVWLLQOQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771403977 (564h8m33.263836419s) [noise=true , address=145.79.227.2:4113 , piece_id=RCMMPOCO42UV5U3FZKGANAT26N7OCM3CJZHV46HLKGHMCWH7376Q, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771458605 (564h8m33.263891048s) [noise=true , address=145.79.227.13:4804 , piece_id=WPGKDLRIVO5TFWTTZU6TJOF6HEEXSR3C6J2K377H3WWSZC5RO3IQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771468222 (564h8m33.263900656s) [noise=true , address=145.79.227.11:4730 , piece_id=JHKHU43WPHKIKYPMJJR3HVM73B6KL2GEBDYBMNKJM5BFAJKK4OQQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771504598 (564h8m33.263937046s) [seg_pos={0 0}, upload=496286]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:16:12.885852769 (564h8m33.378285211s) [upload=496286]: Done uploading segments... Sep 22 02:16:12.885888069 (564h8m33.378320516s) [upload=496286]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.408747584s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................................................XXXXXXX..X] duration: 30.117029151s FAIL (2025-09-22 02:16:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:15:43.060999338 (564h8m3.55343178s) [upload=496286]: Starting upload Sep 22 02:15:43.061082211 (564h8m3.553514654s) [upload=496286]: Done starting upload Sep 22 02:15:43.061220388 (564h8m3.553652823s) [upload=496286]: Uploading segments... Sep 22 02:15:43.061425366 (564h8m3.5538578s) [upload=496286]: Got next segment. Inline: false Sep 22 02:15:43.061501012 (564h8m3.553933464s) [upload=496286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:15:43.061527067 (564h8m3.553959517s) [upload=496286]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:15:43.174021732 (564h8m3.666454178s) [seg_pos={0 0}, upload=496286]: Begin upload segment... Sep 22 02:15:43.174133648 (564h8m3.666566091s) [seg_pos={0 0}, upload=496286]: Done begin upload segment. Sep 22 02:15:43.174162119 (564h8m3.666594563s) [noise=true , address=145.79.227.6:4258 , piece_id=LDCDWV447WOJYW3LPMOECURNXHOBQBS6GZXXA2ETGS6QGI62SPDA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174220457 (564h8m3.666652906s) [noise=true , address=145.79.227.11:4730 , piece_id=JHKHU43WPHKIKYPMJJR3HVM73B6KL2GEBDYBMNKJM5BFAJKK4OQQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174581055 (564h8m3.667013523s) [noise=true , address=45.91.116.4:4939 , piece_id=AJAIWWFDDCHLFRQZ3KIC6UGIZBPYXAERO2AHPUTSLREFLDJVDA7Q, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174728551 (564h8m3.667160996s) [noise=true , address=45.91.116.13:4155 , piece_id=TUAILL2Z76LYEV7JDEXCMGM62EHAOSYYPZJVJ3XMEAN7FC7X54WQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.174882432 (564h8m3.667314878s) [noise=true , address=145.79.227.7:4293 , piece_id=REYRRE5XZNS2RHEJCUU77U335VSQYX7NN4EPS5WTJGITP7RF54JQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.175172953 (564h8m3.667605626s) [noise=true , address=145.79.227.2:4113 , piece_id=RCMMPOCO42UV5U3FZKGANAT26N7OCM3CJZHV46HLKGHMCWH7376Q, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.175517872 (564h8m3.667950317s) [noise=true , address=45.91.116.9:4262 , piece_id=T25DH4GQGBLRBO2CM2KEU23IQJOZQHFUO26ROJUH6VB5TMZ3RZJA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.177522891 (564h8m3.669955342s) [noise=true , address=45.91.116.12:4226 , piece_id=7WIQD2XI3C6NOE6NKHKRDHROHDZLHZERCNS3QH3CG6ZIUFZAZ3KA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.177857355 (564h8m3.670289809s) [noise=true , address=45.91.116.5:4299 , piece_id=DI2YSURBCMM6EVGRPOI7B5BBIUVH3A3TNUVK7CC2CJ5L6OWXNDHQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.178483650 (564h8m3.670916098s) [noise=true , address=145.79.227.13:4804 , piece_id=WPGKDLRIVO5TFWTTZU6TJOF6HEEXSR3C6J2K377H3WWSZC5RO3IQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.178817920 (564h8m3.671250379s) [noise=true , address=145.79.227.14:4838 , piece_id=PZDXWEBW2X6ZYP3HWI6BQL5YY2U7TXFUEB52AVMWIKL2KV3SX43A, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179109436 (564h8m3.671541879s) [noise=true , address=45.91.116.6:4973 , piece_id=AKCKIXSHRTNU6N55QWWB5EW5TS6QB6SELPP4WSLN65KFKYMYU3MA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179392435 (564h8m3.671824881s) [noise=true , address=45.91.116.13:4154 , piece_id=IJOYTF2URZOSWDFYN6KO3P7LITBYU6QIESWFEASHVUGUECMJJEDA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179666709 (564h8m3.672099151s) [noise=true , address=77.90.31.19:4843 , piece_id=J3K3HIPB7N2RUJ5SIJRCRPPWHIVVO62M5UGQD4LRGMBRZADVEOFA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.179956688 (564h8m3.672389136s) [noise=true , address=45.91.116.8:4048 , piece_id=2EUC7VPHEQBDKZZLZ622DCJEMSU2CV22ZKHVIMCZAY6BL7Z4B3MA, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.180280762 (564h8m3.672713215s) [noise=true , address=145.79.227.16:4912 , piece_id=VQT36O7YTR2V3TR7AUZXM5KTV6JLGJRG7NDZFDHIGQX6ZVWLLQOQ, seg_pos={0 0}, upload=496286]: Uploading piece... Sep 22 02:15:43.792732778 (564h8m4.285165211s) [upload=496286]: Next returned nil segment Sep 22 02:15:43.792762497 (564h8m4.285194943s) [upload=496286]: Waiting for error group managing segments... Sep 22 02:15:43.928148341 (564h8m4.420580769s) [noise=true , address=45.91.116.4:4939 , piece_id=AJAIWWFDDCHLFRQZ3KIC6UGIZBPYXAERO2AHPUTSLREFLDJVDA7Q, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.932360246 (564h8m4.42479269s) [noise=true , address=45.91.116.6:4973 , piece_id=AKCKIXSHRTNU6N55QWWB5EW5TS6QB6SELPP4WSLN65KFKYMYU3MA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.934574498 (564h8m4.427006931s) [noise=true , address=45.91.116.13:4155 , piece_id=TUAILL2Z76LYEV7JDEXCMGM62EHAOSYYPZJVJ3XMEAN7FC7X54WQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.936508997 (564h8m4.42894144s) [noise=true , address=45.91.116.9:4262 , piece_id=T25DH4GQGBLRBO2CM2KEU23IQJOZQHFUO26ROJUH6VB5TMZ3RZJA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.940156209 (564h8m4.432588656s) [noise=true , address=77.90.31.19:4843 , piece_id=J3K3HIPB7N2RUJ5SIJRCRPPWHIVVO62M5UGQD4LRGMBRZADVEOFA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.943311504 (564h8m4.435743948s) [noise=true , address=45.91.116.5:4299 , piece_id=DI2YSURBCMM6EVGRPOI7B5BBIUVH3A3TNUVK7CC2CJ5L6OWXNDHQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.946116925 (564h8m4.438549372s) [noise=true , address=45.91.116.13:4154 , piece_id=IJOYTF2URZOSWDFYN6KO3P7LITBYU6QIESWFEASHVUGUECMJJEDA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.947998253 (564h8m4.440430697s) [noise=true , address=45.91.116.8:4048 , piece_id=2EUC7VPHEQBDKZZLZ622DCJEMSU2CV22ZKHVIMCZAY6BL7Z4B3MA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:15:43.957203589 (564h8m4.449636027s) [noise=true , address=45.91.116.12:4226 , piece_id=7WIQD2XI3C6NOE6NKHKRDHROHDZLHZERCNS3QH3CG6ZIUFZAZ3KA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: <nil> Sep 22 02:16:12.770990645 (564h8m33.263423088s) [noise=true , address=145.79.227.14:4838 , piece_id=PZDXWEBW2X6ZYP3HWI6BQL5YY2U7TXFUEB52AVMWIKL2KV3SX43A, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771183282 (564h8m33.263615712s) [noise=true , address=145.79.227.6:4258 , piece_id=LDCDWV447WOJYW3LPMOECURNXHOBQBS6GZXXA2ETGS6QGI62SPDA, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771269437 (564h8m33.263701879s) [noise=true , address=145.79.227.7:4293 , piece_id=REYRRE5XZNS2RHEJCUU77U335VSQYX7NN4EPS5WTJGITP7RF54JQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771339130 (564h8m33.263771573s) [noise=true , address=145.79.227.16:4912 , piece_id=VQT36O7YTR2V3TR7AUZXM5KTV6JLGJRG7NDZFDHIGQX6ZVWLLQOQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771403977 (564h8m33.263836419s) [noise=true , address=145.79.227.2:4113 , piece_id=RCMMPOCO42UV5U3FZKGANAT26N7OCM3CJZHV46HLKGHMCWH7376Q, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771458605 (564h8m33.263891048s) [noise=true , address=145.79.227.13:4804 , piece_id=WPGKDLRIVO5TFWTTZU6TJOF6HEEXSR3C6J2K377H3WWSZC5RO3IQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771468222 (564h8m33.263900656s) [noise=true , address=145.79.227.11:4730 , piece_id=JHKHU43WPHKIKYPMJJR3HVM73B6KL2GEBDYBMNKJM5BFAJKK4OQQ, seg_pos={0 0}, upload=496286]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:16:12.771504598 (564h8m33.263937046s) [seg_pos={0 0}, upload=496286]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:16:12.885852769 (564h8m33.378285211s) [upload=496286]: Done uploading segments... Sep 22 02:16:12.885888069 (564h8m33.378320516s) [upload=496286]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.11706086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................XXXX.X....] duration: 2.826169041s FAIL (2025-09-22 02:15:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 09s) [upload=496229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:32.568355888 (564h3m53.060788338s) [upload=496229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:32.785570279 (564h3m53.278002721s) [seg_pos={0 0}, upload=496229]: Begin upload segment... Sep 22 02:11:32.785631601 (564h3m53.278064043s) [seg_pos={0 0}, upload=496229]: Done begin upload segment. Sep 22 02:11:32.785664405 (564h3m53.278096857s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786022555 (564h3m53.278455008s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786357639 (564h3m53.278790179s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787036442 (564h3m53.279468887s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787554449 (564h3m53.279986906s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787678802 (564h3m53.280111247s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.788704451 (564h3m53.281136902s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789239573 (564h3m53.28167202s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789890088 (564h3m53.28232254s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791104648 (564h3m53.283537099s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791163009 (564h3m53.283595463s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791963527 (564h3m53.284395973s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792233635 (564h3m53.284666084s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792670058 (564h3m53.285102509s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793001018 (564h3m53.285433465s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793065624 (564h3m53.285498076s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793343127 (564h3m53.285775571s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793436315 (564h3m53.285868767s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:33.357284519 (564h3m53.849716948s) [upload=496229]: Next returned nil segment Sep 22 02:11:33.357323433 (564h3m53.849755875s) [upload=496229]: Waiting for error group managing segments... Sep 22 02:11:33.497597512 (564h3m53.990029947s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.502877784 (564h3m53.995310219s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524891843 (564h3m54.017324277s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524980105 (564h3m54.017412541s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.536344372 (564h3m54.028776807s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.541124382 (564h3m54.033556827s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.542730012 (564h3m54.035162447s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.558527714 (564h3m54.050960149s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.577474098 (564h3m54.069906531s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.585252722 (564h3m54.077685156s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.602570911 (564h3m54.095003345s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.621540165 (564h3m54.113972594s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:12:32.277068172 (564h4m52.769500614s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:145.79.227.5:4228): protocol: rpc: dial tcp 145.79.227.5:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277102399 (564h4m52.769534832s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs, address:145.79.227.9:4385): protocol: rpc: dial tcp 145.79.227.9:4385: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277140081 (564h4m52.769572518s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277177501 (564h4m52.769609943s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277196881 (564h4m52.769629316s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:145.79.227.3:4165): protocol: rpc: dial tcp 145.79.227.3:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277240400 (564h4m52.769672831s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:145.79.227.12:4789): protocol: rpc: dial tcp 145.79.227.12:4789: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277275791 (564h4m52.769708224s) [seg_pos={0 0}, upload=496229]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:12:32.392722063 (564h4m52.885154499s) [upload=496229]: Done uploading segments... Sep 22 02:12:32.392762598 (564h4m52.88519503s) [upload=496229]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.826223867s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................................XXXXXXX..] duration: 2.583089563s FAIL (2025-09-22 02:14:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:12:56.368506370 (564h5m16.860938799s) [upload=496247]: Starting upload Sep 22 02:12:56.368569847 (564h5m16.861002299s) [upload=496247]: Done starting upload Sep 22 02:12:56.368805992 (564h5m16.861238427s) [upload=496247]: Uploading segments... Sep 22 02:12:56.368916436 (564h5m16.861348883s) [upload=496247]: Got next segment. Inline: false Sep 22 02:12:56.369048322 (564h5m16.861480763s) [upload=496247]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:56.369057877 (564h5m16.861490327s) [upload=496247]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:56.486062639 (564h5m16.978495087s) [seg_pos={0 0}, upload=496247]: Begin upload segment... Sep 22 02:12:56.486194330 (564h5m16.978626774s) [seg_pos={0 0}, upload=496247]: Done begin upload segment. Sep 22 02:12:56.486229495 (564h5m16.978661947s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486243975 (564h5m16.978676426s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486377388 (564h5m16.978809841s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486724620 (564h5m16.979157064s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486798261 (564h5m16.979230714s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487103849 (564h5m16.979536295s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487782184 (564h5m16.980214631s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488278718 (564h5m16.98071117s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488361904 (564h5m16.980794357s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489386482 (564h5m16.981818936s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489585681 (564h5m16.982018134s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490640573 (564h5m16.983073042s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490850629 (564h5m16.983283079s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491765274 (564h5m16.984197714s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491866190 (564h5m16.984298646s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.502749089 (564h5m16.995181531s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:57.116635507 (564h5m17.609067941s) [upload=496247]: Next returned nil segment Sep 22 02:12:57.116670937 (564h5m17.609103382s) [upload=496247]: Waiting for error group managing segments... Sep 22 02:12:57.244763034 (564h5m17.737195464s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.253619485 (564h5m17.746051913s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.254714247 (564h5m17.747146682s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.258931838 (564h5m17.751364273s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262868617 (564h5m17.755301059s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262946554 (564h5m17.755378983s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266213223 (564h5m17.758645652s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266814085 (564h5m17.759246528s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266870299 (564h5m17.759302727s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.268901178 (564h5m17.761333613s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.273224224 (564h5m17.765656654s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:13:26.065559213 (564h5m46.557991663s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065629896 (564h5m46.558062347s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065792502 (564h5m46.558224943s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065823774 (564h5m46.558256208s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065998102 (564h5m46.558430541s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.066055246 (564h5m46.558487698s) [seg_pos={0 0}, upload=496247]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:13:26.185374108 (564h5m46.67780655s) [upload=496247]: Done uploading segments... Sep 22 02:13:26.185410231 (564h5m46.67784268s) [upload=496247]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.583120715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................................XXXX.X...] duration: 2.64878639s FAIL (2025-09-22 02:14:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 09s) [upload=496229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:32.568355888 (564h3m53.060788338s) [upload=496229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:32.785570279 (564h3m53.278002721s) [seg_pos={0 0}, upload=496229]: Begin upload segment... Sep 22 02:11:32.785631601 (564h3m53.278064043s) [seg_pos={0 0}, upload=496229]: Done begin upload segment. Sep 22 02:11:32.785664405 (564h3m53.278096857s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786022555 (564h3m53.278455008s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786357639 (564h3m53.278790179s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787036442 (564h3m53.279468887s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787554449 (564h3m53.279986906s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787678802 (564h3m53.280111247s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.788704451 (564h3m53.281136902s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789239573 (564h3m53.28167202s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789890088 (564h3m53.28232254s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791104648 (564h3m53.283537099s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791163009 (564h3m53.283595463s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791963527 (564h3m53.284395973s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792233635 (564h3m53.284666084s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792670058 (564h3m53.285102509s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793001018 (564h3m53.285433465s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793065624 (564h3m53.285498076s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793343127 (564h3m53.285775571s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793436315 (564h3m53.285868767s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:33.357284519 (564h3m53.849716948s) [upload=496229]: Next returned nil segment Sep 22 02:11:33.357323433 (564h3m53.849755875s) [upload=496229]: Waiting for error group managing segments... Sep 22 02:11:33.497597512 (564h3m53.990029947s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.502877784 (564h3m53.995310219s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524891843 (564h3m54.017324277s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524980105 (564h3m54.017412541s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.536344372 (564h3m54.028776807s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.541124382 (564h3m54.033556827s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.542730012 (564h3m54.035162447s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.558527714 (564h3m54.050960149s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.577474098 (564h3m54.069906531s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.585252722 (564h3m54.077685156s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.602570911 (564h3m54.095003345s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.621540165 (564h3m54.113972594s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:12:32.277068172 (564h4m52.769500614s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:145.79.227.5:4228): protocol: rpc: dial tcp 145.79.227.5:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277102399 (564h4m52.769534832s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs, address:145.79.227.9:4385): protocol: rpc: dial tcp 145.79.227.9:4385: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277140081 (564h4m52.769572518s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277177501 (564h4m52.769609943s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277196881 (564h4m52.769629316s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:145.79.227.3:4165): protocol: rpc: dial tcp 145.79.227.3:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277240400 (564h4m52.769672831s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:145.79.227.12:4789): protocol: rpc: dial tcp 145.79.227.12:4789: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277275791 (564h4m52.769708224s) [seg_pos={0 0}, upload=496229]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:12:32.392722063 (564h4m52.885154499s) [upload=496229]: Done uploading segments... Sep 22 02:12:32.392762598 (564h4m52.88519503s) [upload=496229]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.648824404s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................................XXXXXXX.] duration: 2.527321867s FAIL (2025-09-22 02:14:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:12:56.368506370 (564h5m16.860938799s) [upload=496247]: Starting upload Sep 22 02:12:56.368569847 (564h5m16.861002299s) [upload=496247]: Done starting upload Sep 22 02:12:56.368805992 (564h5m16.861238427s) [upload=496247]: Uploading segments... Sep 22 02:12:56.368916436 (564h5m16.861348883s) [upload=496247]: Got next segment. Inline: false Sep 22 02:12:56.369048322 (564h5m16.861480763s) [upload=496247]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:56.369057877 (564h5m16.861490327s) [upload=496247]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:56.486062639 (564h5m16.978495087s) [seg_pos={0 0}, upload=496247]: Begin upload segment... Sep 22 02:12:56.486194330 (564h5m16.978626774s) [seg_pos={0 0}, upload=496247]: Done begin upload segment. Sep 22 02:12:56.486229495 (564h5m16.978661947s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486243975 (564h5m16.978676426s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486377388 (564h5m16.978809841s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486724620 (564h5m16.979157064s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486798261 (564h5m16.979230714s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487103849 (564h5m16.979536295s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487782184 (564h5m16.980214631s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488278718 (564h5m16.98071117s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488361904 (564h5m16.980794357s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489386482 (564h5m16.981818936s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489585681 (564h5m16.982018134s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490640573 (564h5m16.983073042s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490850629 (564h5m16.983283079s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491765274 (564h5m16.984197714s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491866190 (564h5m16.984298646s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.502749089 (564h5m16.995181531s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:57.116635507 (564h5m17.609067941s) [upload=496247]: Next returned nil segment Sep 22 02:12:57.116670937 (564h5m17.609103382s) [upload=496247]: Waiting for error group managing segments... Sep 22 02:12:57.244763034 (564h5m17.737195464s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.253619485 (564h5m17.746051913s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.254714247 (564h5m17.747146682s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.258931838 (564h5m17.751364273s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262868617 (564h5m17.755301059s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262946554 (564h5m17.755378983s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266213223 (564h5m17.758645652s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266814085 (564h5m17.759246528s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266870299 (564h5m17.759302727s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.268901178 (564h5m17.761333613s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.273224224 (564h5m17.765656654s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:13:26.065559213 (564h5m46.557991663s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065629896 (564h5m46.558062347s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065792502 (564h5m46.558224943s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065823774 (564h5m46.558256208s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065998102 (564h5m46.558430541s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.066055246 (564h5m46.558487698s) [seg_pos={0 0}, upload=496247]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:13:26.185374108 (564h5m46.67780655s) [upload=496247]: Done uploading segments... Sep 22 02:13:26.185410231 (564h5m46.67784268s) [upload=496247]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.527355672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................................................XXXX.X..] duration: 3.149064976s FAIL (2025-09-22 02:13:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 09s) [upload=496229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:32.568355888 (564h3m53.060788338s) [upload=496229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:32.785570279 (564h3m53.278002721s) [seg_pos={0 0}, upload=496229]: Begin upload segment... Sep 22 02:11:32.785631601 (564h3m53.278064043s) [seg_pos={0 0}, upload=496229]: Done begin upload segment. Sep 22 02:11:32.785664405 (564h3m53.278096857s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786022555 (564h3m53.278455008s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786357639 (564h3m53.278790179s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787036442 (564h3m53.279468887s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787554449 (564h3m53.279986906s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787678802 (564h3m53.280111247s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.788704451 (564h3m53.281136902s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789239573 (564h3m53.28167202s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789890088 (564h3m53.28232254s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791104648 (564h3m53.283537099s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791163009 (564h3m53.283595463s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791963527 (564h3m53.284395973s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792233635 (564h3m53.284666084s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792670058 (564h3m53.285102509s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793001018 (564h3m53.285433465s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793065624 (564h3m53.285498076s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793343127 (564h3m53.285775571s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793436315 (564h3m53.285868767s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:33.357284519 (564h3m53.849716948s) [upload=496229]: Next returned nil segment Sep 22 02:11:33.357323433 (564h3m53.849755875s) [upload=496229]: Waiting for error group managing segments... Sep 22 02:11:33.497597512 (564h3m53.990029947s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.502877784 (564h3m53.995310219s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524891843 (564h3m54.017324277s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524980105 (564h3m54.017412541s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.536344372 (564h3m54.028776807s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.541124382 (564h3m54.033556827s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.542730012 (564h3m54.035162447s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.558527714 (564h3m54.050960149s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.577474098 (564h3m54.069906531s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.585252722 (564h3m54.077685156s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.602570911 (564h3m54.095003345s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.621540165 (564h3m54.113972594s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:12:32.277068172 (564h4m52.769500614s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:145.79.227.5:4228): protocol: rpc: dial tcp 145.79.227.5:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277102399 (564h4m52.769534832s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs, address:145.79.227.9:4385): protocol: rpc: dial tcp 145.79.227.9:4385: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277140081 (564h4m52.769572518s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277177501 (564h4m52.769609943s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277196881 (564h4m52.769629316s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:145.79.227.3:4165): protocol: rpc: dial tcp 145.79.227.3:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277240400 (564h4m52.769672831s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:145.79.227.12:4789): protocol: rpc: dial tcp 145.79.227.12:4789: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277275791 (564h4m52.769708224s) [seg_pos={0 0}, upload=496229]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:12:32.392722063 (564h4m52.885154499s) [upload=496229]: Done uploading segments... Sep 22 02:12:32.392762598 (564h4m52.88519503s) [upload=496229]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 3.149158214s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration: 30.121509337s FAIL (2025-09-22 02:13:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:12:56.368506370 (564h5m16.860938799s) [upload=496247]: Starting upload Sep 22 02:12:56.368569847 (564h5m16.861002299s) [upload=496247]: Done starting upload Sep 22 02:12:56.368805992 (564h5m16.861238427s) [upload=496247]: Uploading segments... Sep 22 02:12:56.368916436 (564h5m16.861348883s) [upload=496247]: Got next segment. Inline: false Sep 22 02:12:56.369048322 (564h5m16.861480763s) [upload=496247]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:12:56.369057877 (564h5m16.861490327s) [upload=496247]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:12:56.486062639 (564h5m16.978495087s) [seg_pos={0 0}, upload=496247]: Begin upload segment... Sep 22 02:12:56.486194330 (564h5m16.978626774s) [seg_pos={0 0}, upload=496247]: Done begin upload segment. Sep 22 02:12:56.486229495 (564h5m16.978661947s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486243975 (564h5m16.978676426s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486377388 (564h5m16.978809841s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486724620 (564h5m16.979157064s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.486798261 (564h5m16.979230714s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487103849 (564h5m16.979536295s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.487782184 (564h5m16.980214631s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488278718 (564h5m16.98071117s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.488361904 (564h5m16.980794357s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489386482 (564h5m16.981818936s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.489585681 (564h5m16.982018134s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490640573 (564h5m16.983073042s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.490850629 (564h5m16.983283079s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491765274 (564h5m16.984197714s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.491866190 (564h5m16.984298646s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:56.502749089 (564h5m16.995181531s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Uploading piece... Sep 22 02:12:57.116635507 (564h5m17.609067941s) [upload=496247]: Next returned nil segment Sep 22 02:12:57.116670937 (564h5m17.609103382s) [upload=496247]: Waiting for error group managing segments... Sep 22 02:12:57.244763034 (564h5m17.737195464s) [noise=true , address=45.91.116.4:4940 , piece_id=5CYPKKRZAGCXMPMZZM3YT5EJBLBPRC5RIO4MTNWFL2PSBM5AET5A, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.253619485 (564h5m17.746051913s) [noise=true , address=45.91.116.13:4153 , piece_id=YNYAYBFQETQV56PB47NT7V4HIDJ4ZIGY3AQR6CWGNU6B5KI4HTYA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.254714247 (564h5m17.747146682s) [noise=true , address=45.91.116.8:4047 , piece_id=3QITI6BJEI4FKNFNOJE55MQYGTOFKUY4JV2CDNXMCLE25TOUUYPA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.258931838 (564h5m17.751364273s) [noise=true , address=45.91.116.5:4297 , piece_id=XUYKQ2OHF3U3S3CN56SLPAB6S2TNRN3SB2U6KX7L2VLOYL2C4RMA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262868617 (564h5m17.755301059s) [noise=true , address=45.91.116.10:4082 , piece_id=R5BKKZZURZ7VFEKXTXR6SEM5V3WP7AA5OSYXUQEGEHQCRHFV74MA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.262946554 (564h5m17.755378983s) [noise=true , address=77.90.31.13:4626 , piece_id=2NYTQEHZFYSBD7WJFZGTJMCYI2UCJLYZQBUSLDSRRFPXDZOUQJLQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266213223 (564h5m17.758645652s) [noise=true , address=77.90.31.14:4661 , piece_id=D42VKFOIVQHCOU5BMPTDHYGIFTKYZAWHTF6K25FMUTXGBU3O7MZQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266814085 (564h5m17.759246528s) [noise=true , address=77.90.31.18:4806 , piece_id=G43PTYBUUKTAGXXBDTZKC6SJ7RPL7ACGXRA24C7OYF74DZYLUDOA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.266870299 (564h5m17.759302727s) [noise=true , address=45.91.116.7:4009 , piece_id=QOIOTYKQBM4NI5HTQBK62Y5UJOGY52VJ7BNWYIPPOY2NM6JTIV7Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.268901178 (564h5m17.761333613s) [noise=true , address=77.90.31.17:4771 , piece_id=YNFNTTV4G4GVVYOHUIFZTYZWKHZF2J3CJ7VJHF7L6EFGWGX62VCQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:12:57.273224224 (564h5m17.765656654s) [noise=true , address=77.90.31.13:4628 , piece_id=TLND2MVXXSETRY4WA2XG353OMVSSOCS7PKMQBSVPFU24A6Q3JAVQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: <nil> Sep 22 02:13:26.065559213 (564h5m46.557991663s) [noise=true , address=145.79.227.17:4970 , piece_id=L6XXUZQZR2UC5F5U5B3TN3CZK46ME64TNPN242S6Q54CGYBR7COQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065629896 (564h5m46.558062347s) [noise=true , address=145.79.227.14:4837 , piece_id=LCFQ7QKY3U2FWGVPBWC3S6KAI6XKGOTCQNWSLB3GDD7XSV26UMSA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065792502 (564h5m46.558224943s) [noise=true , address=145.79.227.5:4221 , piece_id=5JQ5BESHIDINBKBDLZVRUWGL56LB5COOGAZFRDQX6X7HJP6MHPNQ, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065823774 (564h5m46.558256208s) [noise=true , address=145.79.227.4:4186 , piece_id=XS6XXO6VLDMTONZ7UFXLQE4JCXJUEYUXFTLVPKC7G544ZYXTH2PA, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.065998102 (564h5m46.558430541s) [noise=true , address=145.79.227.16:4912 , piece_id=43OQKPFU55LAUDZP3PG27ZMWYRCSRMVMSFQK5XPUMCNYXY2YGR3Q, seg_pos={0 0}, upload=496247]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:13:26.066055246 (564h5m46.558487698s) [seg_pos={0 0}, upload=496247]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:13:26.185374108 (564h5m46.67780655s) [upload=496247]: Done uploading segments... Sep 22 02:13:26.185410231 (564h5m46.67784268s) [upload=496247]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.121545672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................XXXX.X.] duration: 2.896781565s FAIL (2025-09-22 02:12:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 09s) [upload=496229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:32.568355888 (564h3m53.060788338s) [upload=496229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:32.785570279 (564h3m53.278002721s) [seg_pos={0 0}, upload=496229]: Begin upload segment... Sep 22 02:11:32.785631601 (564h3m53.278064043s) [seg_pos={0 0}, upload=496229]: Done begin upload segment. Sep 22 02:11:32.785664405 (564h3m53.278096857s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786022555 (564h3m53.278455008s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786357639 (564h3m53.278790179s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787036442 (564h3m53.279468887s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787554449 (564h3m53.279986906s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787678802 (564h3m53.280111247s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.788704451 (564h3m53.281136902s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789239573 (564h3m53.28167202s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789890088 (564h3m53.28232254s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791104648 (564h3m53.283537099s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791163009 (564h3m53.283595463s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791963527 (564h3m53.284395973s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792233635 (564h3m53.284666084s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792670058 (564h3m53.285102509s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793001018 (564h3m53.285433465s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793065624 (564h3m53.285498076s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793343127 (564h3m53.285775571s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793436315 (564h3m53.285868767s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:33.357284519 (564h3m53.849716948s) [upload=496229]: Next returned nil segment Sep 22 02:11:33.357323433 (564h3m53.849755875s) [upload=496229]: Waiting for error group managing segments... Sep 22 02:11:33.497597512 (564h3m53.990029947s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.502877784 (564h3m53.995310219s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524891843 (564h3m54.017324277s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524980105 (564h3m54.017412541s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.536344372 (564h3m54.028776807s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.541124382 (564h3m54.033556827s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.542730012 (564h3m54.035162447s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.558527714 (564h3m54.050960149s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.577474098 (564h3m54.069906531s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.585252722 (564h3m54.077685156s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.602570911 (564h3m54.095003345s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.621540165 (564h3m54.113972594s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:12:32.277068172 (564h4m52.769500614s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:145.79.227.5:4228): protocol: rpc: dial tcp 145.79.227.5:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277102399 (564h4m52.769534832s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs, address:145.79.227.9:4385): protocol: rpc: dial tcp 145.79.227.9:4385: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277140081 (564h4m52.769572518s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277177501 (564h4m52.769609943s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277196881 (564h4m52.769629316s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:145.79.227.3:4165): protocol: rpc: dial tcp 145.79.227.3:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277240400 (564h4m52.769672831s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:145.79.227.12:4789): protocol: rpc: dial tcp 145.79.227.12:4789: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277275791 (564h4m52.769708224s) [seg_pos={0 0}, upload=496229]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:12:32.392722063 (564h4m52.885154499s) [upload=496229]: Done uploading segments... Sep 22 02:12:32.392762598 (564h4m52.88519503s) [upload=496229]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 2.896808781s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXXX.X] duration: 1m0.118076504s FAIL (2025-09-22 02:12:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 09s) [upload=496229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:32.568355888 (564h3m53.060788338s) [upload=496229]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:32.785570279 (564h3m53.278002721s) [seg_pos={0 0}, upload=496229]: Begin upload segment... Sep 22 02:11:32.785631601 (564h3m53.278064043s) [seg_pos={0 0}, upload=496229]: Done begin upload segment. Sep 22 02:11:32.785664405 (564h3m53.278096857s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786022555 (564h3m53.278455008s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.786357639 (564h3m53.278790179s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787036442 (564h3m53.279468887s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787554449 (564h3m53.279986906s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.787678802 (564h3m53.280111247s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.788704451 (564h3m53.281136902s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789239573 (564h3m53.28167202s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.789890088 (564h3m53.28232254s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791104648 (564h3m53.283537099s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791163009 (564h3m53.283595463s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.791963527 (564h3m53.284395973s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792233635 (564h3m53.284666084s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.792670058 (564h3m53.285102509s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793001018 (564h3m53.285433465s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793065624 (564h3m53.285498076s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793343127 (564h3m53.285775571s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:32.793436315 (564h3m53.285868767s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Uploading piece... Sep 22 02:11:33.357284519 (564h3m53.849716948s) [upload=496229]: Next returned nil segment Sep 22 02:11:33.357323433 (564h3m53.849755875s) [upload=496229]: Waiting for error group managing segments... Sep 22 02:11:33.497597512 (564h3m53.990029947s) [noise=true , address=77.90.31.21:4905 , piece_id=6VWKJAVCFDYN2ZEBMXTPNFW23WSAAJED7QEM4TNVV4BQ7YUYFU4A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.502877784 (564h3m53.995310219s) [noise=true , address=77.90.31.10:4539 , piece_id=HUNPG3GOJ6X4NNY5EK26SJVKXPMY6L7DJMMXOGE7763BRZO7S4DQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524891843 (564h3m54.017324277s) [noise=true , address=77.90.31.19:4861 , piece_id=PI6N2CPXFCZIDPCCZGCV5H7YBJDL4XXVL6IWENWDTREIKJFBVOQA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.524980105 (564h3m54.017412541s) [noise=true , address=77.90.31.13:4660 , piece_id=Y3DKWK3CBD2HXWLXCYYI6RVZAVDGNSGYX6U7S47UCDRAOX47VAMA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.536344372 (564h3m54.028776807s) [noise=true , address=77.90.31.18:4832 , piece_id=NVU2GBKOVFS6R5YKKFVNJ5GOL5ZXFQPCPIAAUVWPZFLF3NVVGL5Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.541124382 (564h3m54.033556827s) [noise=true , address=145.79.227.46:4627 , piece_id=CW62AVLBZ7NVP73UXKXDYINXL2YQ2NTG3KXFQUFTIQRCW2QJG24A, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.542730012 (564h3m54.035162447s) [noise=true , address=145.79.227.45:4250 , piece_id=JFRCXMSYN757LEPAYNHLQ2JOZUMN4QI7GU42KYPHIQ4E5JYUXXXQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.558527714 (564h3m54.050960149s) [noise=true , address=145.79.227.39:4021 , piece_id=JUCJQXP5GO2W372KIG7RB35OGBBU3MFNJYC7HBI6T2XHFF6R6QFQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.577474098 (564h3m54.069906531s) [noise=true , address=145.79.227.49:4340 , piece_id=X3YKACGZIJX3R62AZ3XMWEYXCO24GLBEGAQW63N5ITJ6NFOVIVEA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.585252722 (564h3m54.077685156s) [noise=true , address=145.79.227.21:4476 , piece_id=CROCO5C4UP5UBGBFTDZT64VQSJ34Z7C62HAPTVEE7TP6NH4WWYQQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.602570911 (564h3m54.095003345s) [noise=true , address=145.79.227.52:4457 , piece_id=27BELZP4KHQBKIKU6B4FCQRFRNWJFMLYOCGJKSFFKISE6R3GQKKQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:11:33.621540165 (564h3m54.113972594s) [noise=true , address=145.79.227.43:4162 , piece_id=ECQFWHMR2SZXSUSV3MJYW75HN5D4LEERLGOW3N4GQ2VKV4Y6AQBA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: <nil> Sep 22 02:12:32.277068172 (564h4m52.769500614s) [noise=true , address=145.79.227.5:4228 , piece_id=H6F3OLW33XZ7ZREXOQMW4L5MOA7OTCSSWRE5UMFS56DYH2GTTWCQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12pkADNFaYa2SqFALQdxwf1CfR1ktUwshQkw7iBJzhdhaRYaszF, address:145.79.227.5:4228): protocol: rpc: dial tcp 145.79.227.5:4228: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277102399 (564h4m52.769534832s) [noise=true , address=145.79.227.9:4385 , piece_id=L4FD6Y3B67IQZD54RVA2HEKLV7G55Z3YBQ3P3VSJQF4DVWFL23TQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:123PZh1ieWYoc9U5CRrRzV1ZTpGYmjLmh5CpRy1NRC8wiLkumbs, address:145.79.227.9:4385): protocol: rpc: dial tcp 145.79.227.9:4385: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277140081 (564h4m52.769572518s) [noise=true , address=145.79.227.4:4191 , piece_id=DJOMHW23VJWGOD4JL2GPMW32NMU3MG33EM6RG4BYVBOVCI3MAWRA, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:1sVWXyKPnLTobAhk6GXZTTP67FbyY3XJ1e2svTiUEa3rrghd2q, address:145.79.227.4:4191): protocol: rpc: dial tcp 145.79.227.4:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277177501 (564h4m52.769609943s) [noise=true , address=145.79.227.10:4403 , piece_id=WTTLYMJH32IPLM2XOTJ4SBCEQMQMAZBC7EPDSTPHQWUN5VWSOI7Q, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:145.79.227.10:4403): protocol: rpc: dial tcp 145.79.227.10:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277196881 (564h4m52.769629316s) [noise=true , address=145.79.227.3:4165 , piece_id=FSDVDR4SCQQ3O2K5QMYSUECJZHLUX3SWOYCVJEQ7FHDA54RLI2WQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:13Ksy7YCGV5u3T5Z6WwKtMjFq6Rs4bayzR7mQFyXbSFuJjaH4F, address:145.79.227.3:4165): protocol: rpc: dial tcp 145.79.227.3:4165: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277240400 (564h4m52.769672831s) [noise=true , address=145.79.227.12:4789 , piece_id=OSAWLU2U3YXGMRF7AGYQQJIBT6YKXOWZ7X25QB363BKBYUDQGMHQ, seg_pos={0 0}, upload=496229]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:145.79.227.12:4789): protocol: rpc: dial tcp 145.79.227.12:4789: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:32.277275791 (564h4m52.769708224s) [seg_pos={0 0}, upload=496229]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:12:32.392722063 (564h4m52.885154499s) [upload=496229]: Done uploading segments... Sep 22 02:12:32.392762598 (564h4m52.88519503s) [upload=496229]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.118106963s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.112698518s FAIL (2025-09-22 02:12:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 02:11:59.706854231 (564h4m20.199286659s) [upload=496233]: Starting upload Sep 22 02:11:59.706931557 (564h4m20.199364001s) [upload=496233]: Done starting upload Sep 22 02:11:59.707008871 (564h4m20.199441299s) [upload=496233]: Uploading segments... Sep 22 02:11:59.707300601 (564h4m20.19973303s) [upload=496233]: Got next segment. Inline: false Sep 22 02:11:59.707341431 (564h4m20.199773859s) [upload=496233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:59.707345862 (564h4m20.19977829s) [upload=496233]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:59.893313389 (564h4m20.385745832s) [seg_pos={0 0}, upload=496233]: Begin upload segment... Sep 22 02:11:59.893363936 (564h4m20.38579637s) [seg_pos={0 0}, upload=496233]: Done begin upload segment. Sep 22 02:11:59.893388504 (564h4m20.385820946s) [noise=true , address=77.90.31.18:4807 , piece_id=P36PXTZBN2WKPJDRNLVOBH7QIRVDBJGYVXIJGLLYT2CTE6A3CDAQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.893415119 (564h4m20.385847571s) [noise=true , address=145.79.227.14:4840 , piece_id=YPHL4B6RHDAD7VKSL2LDROVIK34VAHBGKWR6OV3XHJWPUZTTE6XA, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.893831913 (564h4m20.386264366s) [noise=true , address=145.79.227.16:4912 , piece_id=7S62XFNNFWJCAITFNDVVGIJDOVH3BH3OVPN2QOKNB2B63HBZOCOQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.893859992 (564h4m20.386292434s) [noise=true , address=77.90.31.14:4663 , piece_id=EM6V4RYKOVYDCEOXG47D4RTEM6UD2ZJC5DZRVG6M5WGZWDK7VH4A, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.894274067 (564h4m20.386706514s) [noise=true , address=45.91.116.9:4262 , piece_id=CHOPAB3OF5AMSGPBFQW2VWGK4UODTOOGNI5S7LYHTWK7CQ7RZKPA, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.894334821 (564h4m20.386767266s) [noise=true , address=145.79.227.19:4078 , piece_id=N6UIGI7I4MMUXWVCB4SQUQTM4X46FU2EXDO673OJW43N5HXFRMLA, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.894586726 (564h4m20.387019175s) [noise=true , address=45.91.116.11:4119 , piece_id=SCF4BDR6ZCAUAOB6B7BNVKBJKXOU55JMDIMYQQHDFRLTVLCM5QPQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.894736307 (564h4m20.387168749s) [noise=true , address=77.90.31.10:4517 , piece_id=CVAUPFWCAUXECDV355MILIYC2A3V3GW5RVL2YJXG6MIMHTLY45GQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.894894220 (564h4m20.387326668s) [noise=true , address=145.79.227.2:4113 , piece_id=PMR2IUGMIO4QTNC7VZ4FCGYOYEA5FAGLSA6FZ2MR2OXR2LNYZ2QQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895177932 (564h4m20.387610374s) [noise=true , address=145.79.227.11:4729 , piece_id=CPUQ7YVC2FN5BTEJZZRPZO2TUOMM5LMFOH6VRJ2DYGAAOEILGM3A, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895242317 (564h4m20.387674764s) [noise=true , address=45.91.116.12:4227 , piece_id=UZYR6WSRI4UP64AE776BCFHXFNDTBESSLB3QDTC4OWOVQ5VRCTMQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895464516 (564h4m20.387896961s) [noise=true , address=45.91.116.7:4011 , piece_id=P5CQDBCVX36VRWRBXDCTIKXFU6GMFRBC2457SDPPUSFH4XO4G7DQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895554804 (564h4m20.387987249s) [noise=true , address=77.90.31.17:4771 , piece_id=XDYKBYLQQBTHOTIZRGQFYJZ5KSK7C466LUVVVEGHZYYIEOOMIDQQ, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895845134 (564h4m20.388277594s) [noise=true , address=45.91.116.11:4118 , piece_id=I7B5M3FS2QUKXE2W3RUIVQDZHSRU5UC2F53EL55ZFPZSKQ7J254A, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.895866956 (564h4m20.388299422s) [noise=true , address=45.91.116.5:4298 , piece_id=QISNMBUURG52VVUPAXVNE6S23DUBN4RWC4LM7EGURTUQUZWL6K5A, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:11:59.898054720 (564h4m20.390487155s) [noise=true , address=145.79.227.18:4005 , piece_id=S7X6WRSQT24JSJER3FGDGPJIRJ4KT6MKAVEOZ6DA3PFFXND7I4ZA, seg_pos={0 0}, upload=496233]: Uploading piece... Sep 22 02:12:00.522404429 (564h4m21.014836874s) [upload=496233]: Next returned nil segment Sep 22 02:12:00.522425970 (564h4m21.014858419s) [upload=496233]: Waiting for error group managing segments... Sep 22 02:12:00.647443546 (564h4m21.139875982s) [noise=true , address=45.91.116.9:4262 , piece_id=CHOPAB3OF5AMSGPBFQW2VWGK4UODTOOGNI5S7LYHTWK7CQ7RZKPA, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.651598592 (564h4m21.144031029s) [noise=true , address=77.90.31.18:4807 , piece_id=P36PXTZBN2WKPJDRNLVOBH7QIRVDBJGYVXIJGLLYT2CTE6A3CDAQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.651751057 (564h4m21.144183497s) [noise=true , address=77.90.31.10:4517 , piece_id=CVAUPFWCAUXECDV355MILIYC2A3V3GW5RVL2YJXG6MIMHTLY45GQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.654608619 (564h4m21.147041055s) [noise=true , address=45.91.116.7:4011 , piece_id=P5CQDBCVX36VRWRBXDCTIKXFU6GMFRBC2457SDPPUSFH4XO4G7DQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.659918993 (564h4m21.152351445s) [noise=true , address=45.91.116.11:4119 , piece_id=SCF4BDR6ZCAUAOB6B7BNVKBJKXOU55JMDIMYQQHDFRLTVLCM5QPQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.661406752 (564h4m21.153839188s) [noise=true , address=45.91.116.5:4298 , piece_id=QISNMBUURG52VVUPAXVNE6S23DUBN4RWC4LM7EGURTUQUZWL6K5A, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.666741779 (564h4m21.159174215s) [noise=true , address=77.90.31.17:4771 , piece_id=XDYKBYLQQBTHOTIZRGQFYJZ5KSK7C466LUVVVEGHZYYIEOOMIDQQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.667189972 (564h4m21.159622407s) [noise=true , address=77.90.31.14:4663 , piece_id=EM6V4RYKOVYDCEOXG47D4RTEM6UD2ZJC5DZRVG6M5WGZWDK7VH4A, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.667765602 (564h4m21.160198037s) [noise=true , address=45.91.116.12:4227 , piece_id=UZYR6WSRI4UP64AE776BCFHXFNDTBESSLB3QDTC4OWOVQ5VRCTMQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:00.672162627 (564h4m21.164595063s) [noise=true , address=45.91.116.11:4118 , piece_id=I7B5M3FS2QUKXE2W3RUIVQDZHSRU5UC2F53EL55ZFPZSKQ7J254A, seg_pos={0 0}, upload=496233]: Done uploading piece. err: <nil> Sep 22 02:12:29.406800100 (564h4m49.899232543s) [noise=true , address=145.79.227.2:4113 , piece_id=PMR2IUGMIO4QTNC7VZ4FCGYOYEA5FAGLSA6FZ2MR2OXR2LNYZ2QQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:29.406895985 (564h4m49.899328416s) [noise=true , address=145.79.227.19:4078 , piece_id=N6UIGI7I4MMUXWVCB4SQUQTM4X46FU2EXDO673OJW43N5HXFRMLA, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:12:29.406970429 (564h4m49.899402879s) [noise=true , address=145.79.227.16:4912 , piece_id=7S62XFNNFWJCAITFNDVVGIJDOVH3BH3OVPN2QOKNB2B63HBZOCOQ, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:29.407045820 (564h4m49.899478251s) [noise=true , address=145.79.227.14:4840 , piece_id=YPHL4B6RHDAD7VKSL2LDROVIK34VAHBGKWR6OV3XHJWPUZTTE6XA, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:29.407066127 (564h4m49.899498559s) [noise=true , address=145.79.227.11:4729 , piece_id=CPUQ7YVC2FN5BTEJZZRPZO2TUOMM5LMFOH6VRJ2DYGAAOEILGM3A, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:29.407120142 (564h4m49.899552574s) [noise=true , address=145.79.227.18:4005 , piece_id=S7X6WRSQT24JSJER3FGDGPJIRJ4KT6MKAVEOZ6DA3PFFXND7I4ZA, seg_pos={0 0}, upload=496233]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:12:29.407151760 (564h4m49.899584208s) [seg_pos={0 0}, upload=496233]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 02:12:29.517241786 (564h4m50.00967422s) [upload=496233]: Done uploading segments... Sep 22 02:12:29.517271136 (564h4m50.009703579s) [upload=496233]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.112722035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.12972592s FAIL (2025-09-22 02:11:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:11:29.405214352 (564h3m49.897646808s) [upload=496228]: Starting upload Sep 22 02:11:29.405291284 (564h3m49.897723742s) [upload=496228]: Done starting upload Sep 22 02:11:29.405626287 (564h3m49.898058735s) [upload=496228]: Uploading segments... Sep 22 02:11:29.405699508 (564h3m49.898131961s) [upload=496228]: Got next segment. Inline: false Sep 22 02:11:29.405737221 (564h3m49.898169671s) [upload=496228]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:11:29.405745200 (564h3m49.898177653s) [upload=496228]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:11:29.598753752 (564h3m50.091186197s) [seg_pos={0 0}, upload=496228]: Begin upload segment... Sep 22 02:11:29.598812580 (564h3m50.091245029s) [seg_pos={0 0}, upload=496228]: Done begin upload segment. Sep 22 02:11:29.598837247 (564h3m50.09126969s) [noise=true , address=45.91.116.9:4263 , piece_id=AZ4L765TPSA2527ZAAYBSP6WLCMSLZMGYIXAPIUWUKRCXLUJRHOA, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.598849370 (564h3m50.0912818s) [noise=true , address=77.90.31.21:4879 , piece_id=CH5YMU64M4DXOKJQZXFALRUDK7BUUBKFKEKURKKAIBE4NSSRACDQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.599239449 (564h3m50.091671894s) [noise=true , address=145.79.227.5:4222 , piece_id=QO7HV46DQP3FF5SMFX3VXQW27F55M4FZCXIF5ULK4GVKHYHMKZ2A, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.599291229 (564h3m50.091723658s) [noise=true , address=45.91.116.8:4047 , piece_id=QOPHQ2MTKGUTM6ZQ3AI63YPTNNQGNZY6WPVS2C7RQXNKEGFNZNHA, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.600168451 (564h3m50.092600904s) [noise=true , address=77.90.31.10:4520 , piece_id=2FXYZCXWUMWGHWDPHTTJS2VNU3JNSKEQD33UURID3Q6CDOCS676A, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.600179710 (564h3m50.092612155s) [noise=true , address=145.79.227.15:4875 , piece_id=IDDALXVWEIXRTQBF3EIDC4QCGJGCL7LQZLQITN44D7C5KJQCEV7Q, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.601030233 (564h3m50.093462679s) [noise=true , address=77.90.31.18:4806 , piece_id=XOEXWSBEJJEK5CARCSCY4CGXLUJA2OWMQVAMIIYCWJ3OSO543CEQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.601376796 (564h3m50.093809247s) [noise=true , address=45.91.116.11:4118 , piece_id=GVC4KJ76TJ3KDZ2BV62CNJQNSDHQBQY3N64GISQINRPGOPI4H2BQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.602253433 (564h3m50.094685884s) [noise=true , address=45.91.116.13:4153 , piece_id=UPA6THDTJR4K3GOOQTTHN4T7OUY4MX4H4EZPN3B46WQVSFZRHZZQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.602577287 (564h3m50.095009738s) [noise=true , address=145.79.227.4:4186 , piece_id=RYJH6YMV3EDSIL6XSMEJAPNUROJDNGDAULGYIZHVMBBZ2CW7CIVA, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.603417747 (564h3m50.095850192s) [noise=true , address=77.90.31.15:4697 , piece_id=XJZUDTA2ZMPQQ22VVTQ7LEEEXJFP2OMOINTZHBELD2N2M2J6FISQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.603838737 (564h3m50.096271181s) [noise=true , address=45.91.116.13:4155 , piece_id=N4FG2GVHBD3GFTNXDKI6K4TAEAT3Y6MZKZK3NLS666UKEWMRYU4A, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.604169522 (564h3m50.096601975s) [noise=true , address=145.79.227.12:4767 , piece_id=AAPXBXXWH6GIT3PKGAE524A252Y52JQPPHEA3GRK4ZL4PUB4AILQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.604869114 (564h3m50.097301566s) [noise=true , address=45.91.116.6:4975 , piece_id=QHGV7X52M2JYYDTPXEFNVN4UWJFM6CDS5VVGQ6VTF7AGBF4FX3PQ, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.607347160 (564h3m50.099779589s) [noise=true , address=77.90.31.11:4555 , piece_id=SCRQIXLETN7CQRX3C6C6XCXRY52MOXBX6D4CYV4G243FRTYCHNOA, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:29.613255818 (564h3m50.105688254s) [noise=true , address=77.90.31.17:4770 , piece_id=ZBMVUPIVBTCLYATCZ3PH5Z7XAKDBF3MFERQ4VY7CYU5OKPDC662A, seg_pos={0 0}, upload=496228]: Uploading piece... Sep 22 02:11:30.219201439 (564h3m50.711633868s) [upload=496228]: Next returned nil segment Sep 22 02:11:30.219271991 (564h3m50.711704443s) [upload=496228]: Waiting for error group managing segments... Sep 22 02:11:30.348335131 (564h3m50.840767584s) [noise=true , address=45.91.116.8:4047 , piece_id=QOPHQ2MTKGUTM6ZQ3AI63YPTNNQGNZY6WPVS2C7RQXNKEGFNZNHA, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.350373276 (564h3m50.842805705s) [noise=true , address=45.91.116.13:4153 , piece_id=UPA6THDTJR4K3GOOQTTHN4T7OUY4MX4H4EZPN3B46WQVSFZRHZZQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.350432566 (564h3m50.842865003s) [noise=true , address=45.91.116.9:4263 , piece_id=AZ4L765TPSA2527ZAAYBSP6WLCMSLZMGYIXAPIUWUKRCXLUJRHOA, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.352712105 (564h3m50.845144535s) [noise=true , address=77.90.31.21:4879 , piece_id=CH5YMU64M4DXOKJQZXFALRUDK7BUUBKFKEKURKKAIBE4NSSRACDQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.358660853 (564h3m50.851093299s) [noise=true , address=45.91.116.13:4155 , piece_id=N4FG2GVHBD3GFTNXDKI6K4TAEAT3Y6MZKZK3NLS666UKEWMRYU4A, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.358887500 (564h3m50.851319936s) [noise=true , address=45.91.116.11:4118 , piece_id=GVC4KJ76TJ3KDZ2BV62CNJQNSDHQBQY3N64GISQINRPGOPI4H2BQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.359378625 (564h3m50.851811071s) [noise=true , address=45.91.116.6:4975 , piece_id=QHGV7X52M2JYYDTPXEFNVN4UWJFM6CDS5VVGQ6VTF7AGBF4FX3PQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.363367972 (564h3m50.855800417s) [noise=true , address=77.90.31.15:4697 , piece_id=XJZUDTA2ZMPQQ22VVTQ7LEEEXJFP2OMOINTZHBELD2N2M2J6FISQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.365054382 (564h3m50.85748683s) [noise=true , address=77.90.31.11:4555 , piece_id=SCRQIXLETN7CQRX3C6C6XCXRY52MOXBX6D4CYV4G243FRTYCHNOA, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.371931715 (564h3m50.864364146s) [noise=true , address=77.90.31.17:4770 , piece_id=ZBMVUPIVBTCLYATCZ3PH5Z7XAKDBF3MFERQ4VY7CYU5OKPDC662A, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.374069515 (564h3m50.866501952s) [noise=true , address=77.90.31.10:4520 , piece_id=2FXYZCXWUMWGHWDPHTTJS2VNU3JNSKEQD33UURID3Q6CDOCS676A, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:30.382804068 (564h3m50.875236512s) [noise=true , address=77.90.31.18:4806 , piece_id=XOEXWSBEJJEK5CARCSCY4CGXLUJA2OWMQVAMIIYCWJ3OSO543CEQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: <nil> Sep 22 02:11:59.107730072 (564h4m19.600162503s) [noise=true , address=145.79.227.5:4222 , piece_id=QO7HV46DQP3FF5SMFX3VXQW27F55M4FZCXIF5ULK4GVKHYHMKZ2A, seg_pos={0 0}, upload=496228]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:59.107799453 (564h4m19.600231895s) [noise=true , address=145.79.227.15:4875 , piece_id=IDDALXVWEIXRTQBF3EIDC4QCGJGCL7LQZLQITN44D7C5KJQCEV7Q, seg_pos={0 0}, upload=496228]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: rpc: dial tcp 145.79.227.15:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:11:59.108071189 (564h4m19.600503621s) [noise=true , address=145.79.227.4:4186 , piece_id=RYJH6YMV3EDSIL6XSMEJAPNUROJDNGDAULGYIZHVMBBZ2CW7CIVA, seg_pos={0 0}, upload=496228]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:11:59.108132238 (564h4m19.600564667s) [noise=true , address=145.79.227.12:4767 , piece_id=AAPXBXXWH6GIT3PKGAE524A252Y52JQPPHEA3GRK4ZL4PUB4AILQ, seg_pos={0 0}, upload=496228]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:11:59.108164784 (564h4m19.600597235s) [seg_pos={0 0}, upload=496228]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:11:59.233985502 (564h4m19.726417946s) [upload=496228]: Done uploading segments... Sep 22 02:11:59.234025742 (564h4m19.726458192s) [upload=496228]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.129761945s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.112654902s FAIL (2025-09-22 02:10:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Sep 22 02:10:26.978962428 (564h2m47.471394874s) [upload=496212]: Starting upload Sep 22 02:10:26.979064597 (564h2m47.471497048s) [upload=496212]: Done starting upload Sep 22 02:10:26.979193632 (564h2m47.471626065s) [upload=496212]: Uploading segments... Sep 22 02:10:26.979425098 (564h2m47.471857541s) [upload=496212]: Got next segment. Inline: false Sep 22 02:10:26.979484877 (564h2m47.471917326s) [upload=496212]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:10:26.979499149 (564h2m47.471931592s) [upload=496212]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:10:27.097905613 (564h2m47.590338055s) [seg_pos={0 0}, upload=496212]: Begin upload segment... Sep 22 02:10:27.097994898 (564h2m47.59042734s) [seg_pos={0 0}, upload=496212]: Done begin upload segment. Sep 22 02:10:27.098022865 (564h2m47.590455307s) [noise=true , address=45.91.116.12:4226 , piece_id=FVMTLNNGJJKVQPESZP7RL6JJLEZ7FP2L6VV2LZA6EALIIE64ZOSA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.098673605 (564h2m47.591106051s) [noise=true , address=45.91.116.10:4081 , piece_id=L6Z5QWI7ZXJLVCVYDRUSCYPRBJQRTHFM4FHCHOIWMHK2DXLBSAFA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.099062211 (564h2m47.591494664s) [noise=true , address=145.79.227.13:4801 , piece_id=TYBQOEEW3XMYV5VQQFJGE57J24YX5ZUS2WL2QNS5HL35B55LCYEA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.099412036 (564h2m47.59184448s) [noise=true , address=145.79.227.4:4187 , piece_id=CHD7R7U3Z4NW25DBQ6S4YHJKRGNMSZU67ODA42TXZJSDQYDYKUBA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.099751050 (564h2m47.592183505s) [noise=true , address=145.79.227.2:4114 , piece_id=LMAYOJQ3FGYIYATPNP33ZXT7XT6KDHNNWHC2QRZMCJFBGBATT2OQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.100065808 (564h2m47.592498258s) [noise=true , address=145.79.227.16:4911 , piece_id=JBOKT2PY5OD5B32AIMQJQFMTEMCIO26UGPLZCIXHFJIIQUQDK4PQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.100399535 (564h2m47.592831985s) [noise=true , address=77.90.31.18:4807 , piece_id=AUVE3KV7JW4VHS7N3ZITGPREEYZVIMO4SOPPISZOMSUC7T2ODXZA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.100799956 (564h2m47.593232407s) [noise=true , address=145.79.227.4:4186 , piece_id=TIWSJMCDKGYO54DJYAZ472ACJRESIHASXCMIAARFZHVPFC2HA5FQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.101227123 (564h2m47.59365957s) [noise=true , address=145.79.227.14:4840 , piece_id=U6PU3TUNOLNX57I3M3ERGDT7MI27K55NVRJBKEDYC3ICC4KWJTSA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.101613840 (564h2m47.594046291s) [noise=true , address=145.79.227.19:4078 , piece_id=JTJA55BB4FHGEJQAMQ3TJZFHOCGA4XTAM6CJL4MA6BDYV5FWN5XA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.101986655 (564h2m47.594419101s) [noise=true , address=45.91.116.14:4189 , piece_id=DDB4HLRH7K5B6VVJQVLZBKKRF2DONDOZJXIT6HHWJRIVC6A7Y33A, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.102411854 (564h2m47.594844312s) [noise=true , address=77.90.31.13:4628 , piece_id=2QSPIO7AURAS77KGKCRMBVCFX4LV4LZXFLW7AKK4YLDSQZ3GLYHQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.102817685 (564h2m47.595250133s) [noise=true , address=77.90.31.19:4843 , piece_id=47USESHPUFEDCUVHO7DINYAKDQSEHSP44TQJDO3LQDUWIFJCKWPA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.103142413 (564h2m47.595574857s) [noise=true , address=77.90.31.14:4663 , piece_id=YRYQ5GZM6JLNLCVOPKU56BGMMVZRULPZSLFUQKWU63ZQARZPL3AA, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.103476522 (564h2m47.595908969s) [noise=true , address=145.79.227.6:4258 , piece_id=KBP3HP6BU7TILDU54ZYGO7Z2KGDEM64PBUMKS3TA7LVE5VQH4HDQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.103806226 (564h2m47.596238672s) [noise=true , address=77.90.31.12:4589 , piece_id=GTGMC2ERF75RTRLIYKKW5CJCX56RJ24LMZLS6A5ACN746D23D4BQ, seg_pos={0 0}, upload=496212]: Uploading piece... Sep 22 02:10:27.721903465 (564h2m48.214335913s) [upload=496212]: Next returned nil segment Sep 22 02:10:27.721980327 (564h2m48.214412775s) [upload=496212]: Waiting for error group managing segments... Sep 22 02:10:27.859521035 (564h2m48.351953469s) [noise=true , address=77.90.31.18:4807 , piece_id=AUVE3KV7JW4VHS7N3ZITGPREEYZVIMO4SOPPISZOMSUC7T2ODXZA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.859625080 (564h2m48.352057516s) [noise=true , address=45.91.116.10:4081 , piece_id=L6Z5QWI7ZXJLVCVYDRUSCYPRBJQRTHFM4FHCHOIWMHK2DXLBSAFA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.863903092 (564h2m48.356335534s) [noise=true , address=45.91.116.12:4226 , piece_id=FVMTLNNGJJKVQPESZP7RL6JJLEZ7FP2L6VV2LZA6EALIIE64ZOSA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.870830723 (564h2m48.363263174s) [noise=true , address=77.90.31.13:4628 , piece_id=2QSPIO7AURAS77KGKCRMBVCFX4LV4LZXFLW7AKK4YLDSQZ3GLYHQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.874972383 (564h2m48.367404848s) [noise=true , address=77.90.31.19:4843 , piece_id=47USESHPUFEDCUVHO7DINYAKDQSEHSP44TQJDO3LQDUWIFJCKWPA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.877365543 (564h2m48.369798011s) [noise=true , address=77.90.31.14:4663 , piece_id=YRYQ5GZM6JLNLCVOPKU56BGMMVZRULPZSLFUQKWU63ZQARZPL3AA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.884165441 (564h2m48.376597885s) [noise=true , address=45.91.116.14:4189 , piece_id=DDB4HLRH7K5B6VVJQVLZBKKRF2DONDOZJXIT6HHWJRIVC6A7Y33A, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:27.895243426 (564h2m48.387675878s) [noise=true , address=77.90.31.12:4589 , piece_id=GTGMC2ERF75RTRLIYKKW5CJCX56RJ24LMZLS6A5ACN746D23D4BQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: <nil> Sep 22 02:10:56.671787564 (564h3m17.164220009s) [noise=true , address=145.79.227.6:4258 , piece_id=KBP3HP6BU7TILDU54ZYGO7Z2KGDEM64PBUMKS3TA7LVE5VQH4HDQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:56.671823120 (564h3m17.164255558s) [noise=true , address=145.79.227.4:4187 , piece_id=CHD7R7U3Z4NW25DBQ6S4YHJKRGNMSZU67ODA42TXZJSDQYDYKUBA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.671876750 (564h3m17.164309181s) [noise=true , address=145.79.227.19:4078 , piece_id=JTJA55BB4FHGEJQAMQ3TJZFHOCGA4XTAM6CJL4MA6BDYV5FWN5XA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.671914871 (564h3m17.164347314s) [noise=true , address=145.79.227.13:4801 , piece_id=TYBQOEEW3XMYV5VQQFJGE57J24YX5ZUS2WL2QNS5HL35B55LCYEA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.672004679 (564h3m17.164437112s) [noise=true , address=145.79.227.4:4186 , piece_id=TIWSJMCDKGYO54DJYAZ472ACJRESIHASXCMIAARFZHVPFC2HA5FQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:56.672136015 (564h3m17.164568458s) [noise=true , address=145.79.227.14:4840 , piece_id=U6PU3TUNOLNX57I3M3ERGDT7MI27K55NVRJBKEDYC3ICC4KWJTSA, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.672187262 (564h3m17.164619707s) [noise=true , address=145.79.227.16:4911 , piece_id=JBOKT2PY5OD5B32AIMQJQFMTEMCIO26UGPLZCIXHFJIIQUQDK4PQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.672197851 (564h3m17.164630297s) [noise=true , address=145.79.227.2:4114 , piece_id=LMAYOJQ3FGYIYATPNP33ZXT7XT6KDHNNWHC2QRZMCJFBGBATT2OQ, seg_pos={0 0}, upload=496212]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:56.672232525 (564h3m17.164664972s) [seg_pos={0 0}, upload=496212]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 02:10:56.782709505 (564h3m17.275141946s) [upload=496212]: Done uploading segments... Sep 22 02:10:56.782741599 (564h3m17.275174042s) [upload=496212]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.112682498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 2.877480176s FAIL (2025-09-22 02:10:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== SegmentBegin Sep 22 02:09:52.712216391 (564h2m13.204648843s) [seg_pos={0 0}, upload=496204]: Begin upload segment... Sep 22 02:09:52.712342575 (564h2m13.204775028s) [noise=true , address=145.79.227.41:4088 , piece_id=N4GA5IAYZS3SFALNMJ6NTJPSJ6HKWGDJPI7K4RT6OGXMNJ6CLMEA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.712848436 (564h2m13.205280886s) [noise=true , address=145.79.227.16:4928 , piece_id=ZH5WQBDBCLGLEKYRVNJBXWV2TNQ3KYP6M4ZPVPYKK2QZ6CHKTWWQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.713279718 (564h2m13.205712165s) [noise=true , address=77.90.31.16:4766 , piece_id=4EZ4QZ3VDWC73VZNBHFDIFUXOZ5VUK4MH4UG2CH6ZACAXVUNKMJA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.713683754 (564h2m13.206116207s) [noise=true , address=145.79.227.2:4145 , piece_id=GFL3KYJHZFD6WS5C5UMCZNV7HVGLNFIWEHTULXEV6SJKDE5N3LBA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714035288 (564h2m13.206467731s) [seg_pos={0 0}, upload=496204]: Done begin upload segment. Sep 22 02:09:52.714090125 (564h2m13.206522585s) [noise=true , address=145.79.227.15:4885 , piece_id=N2SPUZWVDL52LE73XGV43I6GJYC5HMIQRASE3RIPTGBFFX4ZUFTA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714390657 (564h2m13.206823106s) [noise=true , address=145.79.227.38:4010 , piece_id=BQD45T76UON4CWRHEGH7SFVTCFPG4KKMICWRE5FGG7WEXMFURVXA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714580540 (564h2m13.207012992s) [noise=true , address=145.79.227.49:4347 , piece_id=CANCTUIDZT5OCZRJQYPW5R2CRO7ESSERCZEZDGJYKUNABDHPRFUQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714778543 (564h2m13.207210997s) [noise=true , address=145.79.227.33:4895 , piece_id=ZLTMZPYSGT5VSZ7FOEWMRD5K4KPCVUZQ65OKGW5HYSANZFPDCWJQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715028075 (564h2m13.207460523s) [noise=true , address=77.90.31.12:4606 , piece_id=6JHRR52VCG4CYNP5QRQJ7M5BMQNCEMZS7W2OSN3X2POAWXTGA5AA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715215594 (564h2m13.207648043s) [noise=true , address=145.79.227.12:4799 , piece_id=6W2JWU2ICHXXA7ZS3G3VNMYB3ZJRFLU2A3KZAPX3SPTIPTYHZESQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715420297 (564h2m13.207852755s) [noise=true , address=145.79.227.11:4757 , piece_id=MQ6DBYP26Q7DQ5QFISBTBMRPGCPMSFJ4JVONFSMGRZGTB54JDSAQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715632300 (564h2m13.208064749s) [noise=true , address=145.79.227.46:4602 , piece_id=XE435LSJ23LSYQARWDS5A5A2V27CD4TUVA4EVARXPHEDJEZHMIVQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715755267 (564h2m13.208187712s) [noise=true , address=145.79.227.43:4172 , piece_id=G6GPBX74RUUTU3PSZR672HDBAX2OMBTCXMUQYR3EWZ57ZJRP34LA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716062436 (564h2m13.208494865s) [noise=true , address=145.79.227.23:4546 , piece_id=2DJULMJ3FAZGSPHYJ5PCMLOVKTYGZQ4ILEF7PT7OHE5LK5WNXYQA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716078478 (564h2m13.208510931s) [noise=true , address=145.79.227.13:4812 , piece_id=PFBMH2A35OIISUOUXLIQBTGIMPSS34JAP3UOVQS7IJZDCA3IQZJA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716377154 (564h2m13.208809602s) [noise=true , address=145.79.227.38:4012 , piece_id=X4RIIF4IF5ANHSV2HVIVXVN2CVZK2QY7NKVCWPUAQJZEUME5HGMA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716689282 (564h2m13.209121757s) [noise=true , address=145.79.227.32:4867 , piece_id=764TQUJOHXY7AI7Q6FLSNDPNOLP4UQ5ORXFFSLI3RYMTQQO2QN3A, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.717045129 (564h2m13.209477577s) [noise=true , address=145.79.227.18:4017 , piece_id=B2LRXFD53AWPBCOIHNYORPZZOHJNDN5OPW2AUB45GZCBCZEWFBZQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:53.279234168 (564h2m13.771666611s) [upload=496204]: Next returned nil segment Sep 22 02:09:53.279255115 (564h2m13.771687561s) [upload=496204]: Waiting for error group managing segments... Sep 22 02:09:53.416692769 (564h2m13.909125229s) [noise=true , address=77.90.31.16:4766 , piece_id=4EZ4QZ3VDWC73VZNBHFDIFUXOZ5VUK4MH4UG2CH6ZACAXVUNKMJA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.443538261 (564h2m13.935970693s) [noise=true , address=77.90.31.12:4606 , piece_id=6JHRR52VCG4CYNP5QRQJ7M5BMQNCEMZS7W2OSN3X2POAWXTGA5AA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.465014907 (564h2m13.95744736s) [noise=true , address=145.79.227.49:4347 , piece_id=CANCTUIDZT5OCZRJQYPW5R2CRO7ESSERCZEZDGJYKUNABDHPRFUQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.466785983 (564h2m13.959218427s) [noise=true , address=145.79.227.38:4010 , piece_id=BQD45T76UON4CWRHEGH7SFVTCFPG4KKMICWRE5FGG7WEXMFURVXA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.467010115 (564h2m13.959442551s) [noise=true , address=145.79.227.33:4895 , piece_id=ZLTMZPYSGT5VSZ7FOEWMRD5K4KPCVUZQ65OKGW5HYSANZFPDCWJQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.467301223 (564h2m13.959733652s) [noise=true , address=145.79.227.38:4012 , piece_id=X4RIIF4IF5ANHSV2HVIVXVN2CVZK2QY7NKVCWPUAQJZEUME5HGMA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.480233302 (564h2m13.972665784s) [noise=true , address=145.79.227.23:4546 , piece_id=2DJULMJ3FAZGSPHYJ5PCMLOVKTYGZQ4ILEF7PT7OHE5LK5WNXYQA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.482812596 (564h2m13.975245032s) [noise=true , address=145.79.227.46:4602 , piece_id=XE435LSJ23LSYQARWDS5A5A2V27CD4TUVA4EVARXPHEDJEZHMIVQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.487287567 (564h2m13.979720012s) [noise=true , address=145.79.227.41:4088 , piece_id=N4GA5IAYZS3SFALNMJ6NTJPSJ6HKWGDJPI7K4RT6OGXMNJ6CLMEA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.491566752 (564h2m13.983999184s) [noise=true , address=145.79.227.43:4172 , piece_id=G6GPBX74RUUTU3PSZR672HDBAX2OMBTCXMUQYR3EWZ57ZJRP34LA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.663758434 (564h2m14.156190869s) [noise=true , address=145.79.227.32:4867 , piece_id=764TQUJOHXY7AI7Q6FLSNDPNOLP4UQ5ORXFFSLI3RYMTQQO2QN3A, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:10:52.188885813 (564h3m12.681318266s) [noise=true , address=145.79.227.15:4885 , piece_id=N2SPUZWVDL52LE73XGV43I6GJYC5HMIQRASE3RIPTGBFFX4ZUFTA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12rLd1AWyRtEVFmK4iJCebp5VAJEjnmhhgtUQcV1FB8vMvRnUhm, address:145.79.227.15:4885): protocol: rpc: dial tcp 145.79.227.15:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.188974102 (564h3m12.681406548s) [noise=true , address=145.79.227.11:4757 , piece_id=MQ6DBYP26Q7DQ5QFISBTBMRPGCPMSFJ4JVONFSMGRZGTB54JDSAQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189042013 (564h3m12.681474446s) [noise=true , address=145.79.227.13:4812 , piece_id=PFBMH2A35OIISUOUXLIQBTGIMPSS34JAP3UOVQS7IJZDCA3IQZJA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12gTRhEnzLoKmQYRttr4hwiTbJ4y3yqMUS1Qg7F2yohb81sd5KV, address:145.79.227.13:4812): protocol: rpc: dial tcp 145.79.227.13:4812: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189096883 (564h3m12.681529327s) [noise=true , address=145.79.227.2:4145 , piece_id=GFL3KYJHZFD6WS5C5UMCZNV7HVGLNFIWEHTULXEV6SJKDE5N3LBA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12sTUH6Y4HBG8ocrPAuuFJiVrcAm1nhpNhJXLW4HaCMdtYmyukp, address:145.79.227.2:4145): protocol: rpc: dial tcp 145.79.227.2:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189155723 (564h3m12.681588171s) [noise=true , address=145.79.227.18:4017 , piece_id=B2LRXFD53AWPBCOIHNYORPZZOHJNDN5OPW2AUB45GZCBCZEWFBZQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8, address:145.79.227.18:4017): protocol: rpc: dial tcp 145.79.227.18:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189205349 (564h3m12.681637799s) [noise=true , address=145.79.227.16:4928 , piece_id=ZH5WQBDBCLGLEKYRVNJBXWV2TNQ3KYP6M4ZPVPYKK2QZ6CHKTWWQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:1fKSEPa36EtKVhzFZ2tCsQVi62RynY5Bwup1kQE17UB1bswsHg, address:145.79.227.16:4928): protocol: rpc: dial tcp 145.79.227.16:4928: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189276707 (564h3m12.681709153s) [noise=true , address=145.79.227.12:4799 , piece_id=6W2JWU2ICHXXA7ZS3G3VNMYB3ZJRFLU2A3KZAPX3SPTIPTYHZESQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:145.79.227.12:4799): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:52.189297910 (564h3m12.681730363s) [seg_pos={0 0}, upload=496204]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:10:52.298889155 (564h3m12.791321589s) [upload=496204]: Done uploading segments... Sep 22 02:10:52.298917963 (564h3m12.791350406s) [upload=496204]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 2.877503112s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.112786958s FAIL (2025-09-22 02:10:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== SegmentBegin Sep 22 02:09:52.712216391 (564h2m13.204648843s) [seg_pos={0 0}, upload=496204]: Begin upload segment... Sep 22 02:09:52.712342575 (564h2m13.204775028s) [noise=true , address=145.79.227.41:4088 , piece_id=N4GA5IAYZS3SFALNMJ6NTJPSJ6HKWGDJPI7K4RT6OGXMNJ6CLMEA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.712848436 (564h2m13.205280886s) [noise=true , address=145.79.227.16:4928 , piece_id=ZH5WQBDBCLGLEKYRVNJBXWV2TNQ3KYP6M4ZPVPYKK2QZ6CHKTWWQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.713279718 (564h2m13.205712165s) [noise=true , address=77.90.31.16:4766 , piece_id=4EZ4QZ3VDWC73VZNBHFDIFUXOZ5VUK4MH4UG2CH6ZACAXVUNKMJA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.713683754 (564h2m13.206116207s) [noise=true , address=145.79.227.2:4145 , piece_id=GFL3KYJHZFD6WS5C5UMCZNV7HVGLNFIWEHTULXEV6SJKDE5N3LBA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714035288 (564h2m13.206467731s) [seg_pos={0 0}, upload=496204]: Done begin upload segment. Sep 22 02:09:52.714090125 (564h2m13.206522585s) [noise=true , address=145.79.227.15:4885 , piece_id=N2SPUZWVDL52LE73XGV43I6GJYC5HMIQRASE3RIPTGBFFX4ZUFTA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714390657 (564h2m13.206823106s) [noise=true , address=145.79.227.38:4010 , piece_id=BQD45T76UON4CWRHEGH7SFVTCFPG4KKMICWRE5FGG7WEXMFURVXA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714580540 (564h2m13.207012992s) [noise=true , address=145.79.227.49:4347 , piece_id=CANCTUIDZT5OCZRJQYPW5R2CRO7ESSERCZEZDGJYKUNABDHPRFUQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.714778543 (564h2m13.207210997s) [noise=true , address=145.79.227.33:4895 , piece_id=ZLTMZPYSGT5VSZ7FOEWMRD5K4KPCVUZQ65OKGW5HYSANZFPDCWJQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715028075 (564h2m13.207460523s) [noise=true , address=77.90.31.12:4606 , piece_id=6JHRR52VCG4CYNP5QRQJ7M5BMQNCEMZS7W2OSN3X2POAWXTGA5AA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715215594 (564h2m13.207648043s) [noise=true , address=145.79.227.12:4799 , piece_id=6W2JWU2ICHXXA7ZS3G3VNMYB3ZJRFLU2A3KZAPX3SPTIPTYHZESQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715420297 (564h2m13.207852755s) [noise=true , address=145.79.227.11:4757 , piece_id=MQ6DBYP26Q7DQ5QFISBTBMRPGCPMSFJ4JVONFSMGRZGTB54JDSAQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715632300 (564h2m13.208064749s) [noise=true , address=145.79.227.46:4602 , piece_id=XE435LSJ23LSYQARWDS5A5A2V27CD4TUVA4EVARXPHEDJEZHMIVQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.715755267 (564h2m13.208187712s) [noise=true , address=145.79.227.43:4172 , piece_id=G6GPBX74RUUTU3PSZR672HDBAX2OMBTCXMUQYR3EWZ57ZJRP34LA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716062436 (564h2m13.208494865s) [noise=true , address=145.79.227.23:4546 , piece_id=2DJULMJ3FAZGSPHYJ5PCMLOVKTYGZQ4ILEF7PT7OHE5LK5WNXYQA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716078478 (564h2m13.208510931s) [noise=true , address=145.79.227.13:4812 , piece_id=PFBMH2A35OIISUOUXLIQBTGIMPSS34JAP3UOVQS7IJZDCA3IQZJA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716377154 (564h2m13.208809602s) [noise=true , address=145.79.227.38:4012 , piece_id=X4RIIF4IF5ANHSV2HVIVXVN2CVZK2QY7NKVCWPUAQJZEUME5HGMA, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.716689282 (564h2m13.209121757s) [noise=true , address=145.79.227.32:4867 , piece_id=764TQUJOHXY7AI7Q6FLSNDPNOLP4UQ5ORXFFSLI3RYMTQQO2QN3A, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:52.717045129 (564h2m13.209477577s) [noise=true , address=145.79.227.18:4017 , piece_id=B2LRXFD53AWPBCOIHNYORPZZOHJNDN5OPW2AUB45GZCBCZEWFBZQ, seg_pos={0 0}, upload=496204]: Uploading piece... Sep 22 02:09:53.279234168 (564h2m13.771666611s) [upload=496204]: Next returned nil segment Sep 22 02:09:53.279255115 (564h2m13.771687561s) [upload=496204]: Waiting for error group managing segments... Sep 22 02:09:53.416692769 (564h2m13.909125229s) [noise=true , address=77.90.31.16:4766 , piece_id=4EZ4QZ3VDWC73VZNBHFDIFUXOZ5VUK4MH4UG2CH6ZACAXVUNKMJA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.443538261 (564h2m13.935970693s) [noise=true , address=77.90.31.12:4606 , piece_id=6JHRR52VCG4CYNP5QRQJ7M5BMQNCEMZS7W2OSN3X2POAWXTGA5AA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.465014907 (564h2m13.95744736s) [noise=true , address=145.79.227.49:4347 , piece_id=CANCTUIDZT5OCZRJQYPW5R2CRO7ESSERCZEZDGJYKUNABDHPRFUQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.466785983 (564h2m13.959218427s) [noise=true , address=145.79.227.38:4010 , piece_id=BQD45T76UON4CWRHEGH7SFVTCFPG4KKMICWRE5FGG7WEXMFURVXA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.467010115 (564h2m13.959442551s) [noise=true , address=145.79.227.33:4895 , piece_id=ZLTMZPYSGT5VSZ7FOEWMRD5K4KPCVUZQ65OKGW5HYSANZFPDCWJQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.467301223 (564h2m13.959733652s) [noise=true , address=145.79.227.38:4012 , piece_id=X4RIIF4IF5ANHSV2HVIVXVN2CVZK2QY7NKVCWPUAQJZEUME5HGMA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.480233302 (564h2m13.972665784s) [noise=true , address=145.79.227.23:4546 , piece_id=2DJULMJ3FAZGSPHYJ5PCMLOVKTYGZQ4ILEF7PT7OHE5LK5WNXYQA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.482812596 (564h2m13.975245032s) [noise=true , address=145.79.227.46:4602 , piece_id=XE435LSJ23LSYQARWDS5A5A2V27CD4TUVA4EVARXPHEDJEZHMIVQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.487287567 (564h2m13.979720012s) [noise=true , address=145.79.227.41:4088 , piece_id=N4GA5IAYZS3SFALNMJ6NTJPSJ6HKWGDJPI7K4RT6OGXMNJ6CLMEA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.491566752 (564h2m13.983999184s) [noise=true , address=145.79.227.43:4172 , piece_id=G6GPBX74RUUTU3PSZR672HDBAX2OMBTCXMUQYR3EWZ57ZJRP34LA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:09:53.663758434 (564h2m14.156190869s) [noise=true , address=145.79.227.32:4867 , piece_id=764TQUJOHXY7AI7Q6FLSNDPNOLP4UQ5ORXFFSLI3RYMTQQO2QN3A, seg_pos={0 0}, upload=496204]: Done uploading piece. err: <nil> Sep 22 02:10:52.188885813 (564h3m12.681318266s) [noise=true , address=145.79.227.15:4885 , piece_id=N2SPUZWVDL52LE73XGV43I6GJYC5HMIQRASE3RIPTGBFFX4ZUFTA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12rLd1AWyRtEVFmK4iJCebp5VAJEjnmhhgtUQcV1FB8vMvRnUhm, address:145.79.227.15:4885): protocol: rpc: dial tcp 145.79.227.15:4885: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.188974102 (564h3m12.681406548s) [noise=true , address=145.79.227.11:4757 , piece_id=MQ6DBYP26Q7DQ5QFISBTBMRPGCPMSFJ4JVONFSMGRZGTB54JDSAQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189042013 (564h3m12.681474446s) [noise=true , address=145.79.227.13:4812 , piece_id=PFBMH2A35OIISUOUXLIQBTGIMPSS34JAP3UOVQS7IJZDCA3IQZJA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12gTRhEnzLoKmQYRttr4hwiTbJ4y3yqMUS1Qg7F2yohb81sd5KV, address:145.79.227.13:4812): protocol: rpc: dial tcp 145.79.227.13:4812: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189096883 (564h3m12.681529327s) [noise=true , address=145.79.227.2:4145 , piece_id=GFL3KYJHZFD6WS5C5UMCZNV7HVGLNFIWEHTULXEV6SJKDE5N3LBA, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12sTUH6Y4HBG8ocrPAuuFJiVrcAm1nhpNhJXLW4HaCMdtYmyukp, address:145.79.227.2:4145): protocol: rpc: dial tcp 145.79.227.2:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189155723 (564h3m12.681588171s) [noise=true , address=145.79.227.18:4017 , piece_id=B2LRXFD53AWPBCOIHNYORPZZOHJNDN5OPW2AUB45GZCBCZEWFBZQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12wS9YpUnsjgkdTPSowozp8txmqAhfF4n8s7JVpLb2tdRgQeLw8, address:145.79.227.18:4017): protocol: rpc: dial tcp 145.79.227.18:4017: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189205349 (564h3m12.681637799s) [noise=true , address=145.79.227.16:4928 , piece_id=ZH5WQBDBCLGLEKYRVNJBXWV2TNQ3KYP6M4ZPVPYKK2QZ6CHKTWWQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:1fKSEPa36EtKVhzFZ2tCsQVi62RynY5Bwup1kQE17UB1bswsHg, address:145.79.227.16:4928): protocol: rpc: dial tcp 145.79.227.16:4928: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:52.189276707 (564h3m12.681709153s) [noise=true , address=145.79.227.12:4799 , piece_id=6W2JWU2ICHXXA7ZS3G3VNMYB3ZJRFLU2A3KZAPX3SPTIPTYHZESQ, seg_pos={0 0}, upload=496204]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:145.79.227.12:4799): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:10:52.189297910 (564h3m12.681730363s) [seg_pos={0 0}, upload=496204]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Sep 22 02:10:52.298889155 (564h3m12.791321589s) [upload=496204]: Done uploading segments... Sep 22 02:10:52.298917963 (564h3m12.791350406s) [upload=496204]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.112812186s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.108446509s FAIL (2025-09-22 02:10:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 22 02:09:48.414090259 (564h2m8.906522712s) [upload=496203]: Starting upload Sep 22 02:09:48.414274214 (564h2m8.90670666s) [upload=496203]: Done starting upload Sep 22 02:09:48.414703701 (564h2m8.907136146s) [upload=496203]: Uploading segments... Sep 22 02:09:48.414779114 (564h2m8.90721155s) [upload=496203]: Got next segment. Inline: false Sep 22 02:09:48.414827192 (564h2m8.907259624s) [upload=496203]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:09:48.414833077 (564h2m8.907265519s) [upload=496203]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:09:48.526922911 (564h2m9.019355355s) [seg_pos={0 0}, upload=496203]: Begin upload segment... Sep 22 02:09:48.527025788 (564h2m9.019458241s) [noise=true , address=77.90.31.14:4662 , piece_id=3KKVUXIVVU6YYVPHD4TSMO2ZOI7ABRVXEP77JS462ADX2EEYHTKA, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.527825298 (564h2m9.020257751s) [noise=true , address=45.91.116.14:4190 , piece_id=BEW7ZRYBTNXANKNGKW6NBSI46GTER23ICT7HUNPMLXG3VXJWT6CA, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.528642812 (564h2m9.021075251s) [seg_pos={0 0}, upload=496203]: Done begin upload segment. Sep 22 02:09:48.528695880 (564h2m9.021128331s) [noise=true , address=77.90.31.17:4770 , piece_id=5EA5VEZ4FGDQV24HZVY4VCRXC5WQ4FGQX3OPTVFQD754LELNVKJQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.529240070 (564h2m9.021672519s) [noise=true , address=45.91.116.13:4155 , piece_id=36JZ24BCODORUXOCXT7PQNDKMOY27B3NGQTFEFOM3X75SL22LIIQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.529767330 (564h2m9.022199778s) [noise=true , address=145.79.227.4:4187 , piece_id=DTUG63TSKA4F6PVSYTEW62Z466EU6JL3D7AIZ3S4XW26PE2QD2JA, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.530852766 (564h2m9.023285219s) [noise=true , address=45.91.116.12:4226 , piece_id=LK4ZPRUT6ADH75RRYQQTZPLF5TXUCNLK2LEQV5YTPNPJD4LHSV7Q, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.531517899 (564h2m9.023950351s) [noise=true , address=77.90.31.12:4589 , piece_id=CAPXMAPOSI4QSHDUQGS7OWXIXL3M2Y2QV3JUTG46HQF37SMDJ3OQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.531977865 (564h2m9.024410317s) [noise=true , address=45.91.116.8:4047 , piece_id=ZLEPJDJHLWC7YX4NW5R2T53OZV6G7J6D2IHPRDFG6DOQ2KN2O4QQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.532489403 (564h2m9.024921855s) [noise=true , address=45.91.116.11:4119 , piece_id=5ITGIU7LIV34BOVE7IF2DWC3DNWTNL27M5OM66GBQTHJKKTNGNSQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.532916948 (564h2m9.025349396s) [noise=true , address=145.79.227.14:4838 , piece_id=G33CKDP4E47EU5Q5PBV3YMYKALGDO5UKDGO53FV75FEKPLJMAWMQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.533390743 (564h2m9.025823189s) [noise=true , address=145.79.227.12:4765 , piece_id=7CVNMSIETGVFB2HGTSSED5D2K5H7TRBS2S26DQA2GJBJRMNEIEPQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.533835371 (564h2m9.026267824s) [noise=true , address=77.90.31.15:4698 , piece_id=WSMCM6XVVHTWPE6AJBMTCSL5AG45CP34QCJCKIUOIUOVULUW7RCQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.534343731 (564h2m9.026776201s) [noise=true , address=45.91.116.6:4974 , piece_id=UDKCYEGJDYMC7KXOHIHL2E4FWMEOHZUUZQXYVFMD56YNZHZKCPZA, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.534809115 (564h2m9.027241568s) [noise=true , address=77.90.31.17:4772 , piece_id=F3WJBFAQM2CCQAIE5C23ZCDRZNWU5DD64VZHY7UHTD367PIYGESA, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.535359489 (564h2m9.027791938s) [noise=true , address=145.79.227.4:4186 , piece_id=WIQBBJOJSIBAZ6SR7YXFNOPM3EFE3SFJ3SN5TOVMNYWMPL5KFU3A, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:48.535811180 (564h2m9.028243633s) [noise=true , address=145.79.227.3:4149 , piece_id=IK57ENL35QM46S453T6BOLGTMES47QH65H4OKY226XUHR5RLBIPQ, seg_pos={0 0}, upload=496203]: Uploading piece... Sep 22 02:09:49.137177292 (564h2m9.629609739s) [upload=496203]: Next returned nil segment Sep 22 02:09:49.137195666 (564h2m9.629628111s) [upload=496203]: Waiting for error group managing segments... Sep 22 02:09:49.273808514 (564h2m9.766240958s) [noise=true , address=45.91.116.13:4155 , piece_id=36JZ24BCODORUXOCXT7PQNDKMOY27B3NGQTFEFOM3X75SL22LIIQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.276176929 (564h2m9.768609358s) [noise=true , address=45.91.116.8:4047 , piece_id=ZLEPJDJHLWC7YX4NW5R2T53OZV6G7J6D2IHPRDFG6DOQ2KN2O4QQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.278129384 (564h2m9.770561841s) [noise=true , address=45.91.116.14:4190 , piece_id=BEW7ZRYBTNXANKNGKW6NBSI46GTER23ICT7HUNPMLXG3VXJWT6CA, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.278969425 (564h2m9.771401867s) [noise=true , address=45.91.116.11:4119 , piece_id=5ITGIU7LIV34BOVE7IF2DWC3DNWTNL27M5OM66GBQTHJKKTNGNSQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.279091783 (564h2m9.771524226s) [noise=true , address=77.90.31.17:4770 , piece_id=5EA5VEZ4FGDQV24HZVY4VCRXC5WQ4FGQX3OPTVFQD754LELNVKJQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.286330761 (564h2m9.778763198s) [noise=true , address=45.91.116.6:4974 , piece_id=UDKCYEGJDYMC7KXOHIHL2E4FWMEOHZUUZQXYVFMD56YNZHZKCPZA, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.286667262 (564h2m9.779099698s) [noise=true , address=77.90.31.14:4662 , piece_id=3KKVUXIVVU6YYVPHD4TSMO2ZOI7ABRVXEP77JS462ADX2EEYHTKA, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.287690742 (564h2m9.78012317s) [noise=true , address=77.90.31.12:4589 , piece_id=CAPXMAPOSI4QSHDUQGS7OWXIXL3M2Y2QV3JUTG46HQF37SMDJ3OQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.290973772 (564h2m9.78340621s) [noise=true , address=77.90.31.17:4772 , piece_id=F3WJBFAQM2CCQAIE5C23ZCDRZNWU5DD64VZHY7UHTD367PIYGESA, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.293754568 (564h2m9.786187s) [noise=true , address=45.91.116.12:4226 , piece_id=LK4ZPRUT6ADH75RRYQQTZPLF5TXUCNLK2LEQV5YTPNPJD4LHSV7Q, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:09:49.303317519 (564h2m9.795749964s) [noise=true , address=77.90.31.15:4698 , piece_id=WSMCM6XVVHTWPE6AJBMTCSL5AG45CP34QCJCKIUOIUOVULUW7RCQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: <nil> Sep 22 02:10:18.128148109 (564h2m38.62058056s) [noise=true , address=145.79.227.3:4149 , piece_id=IK57ENL35QM46S453T6BOLGTMES47QH65H4OKY226XUHR5RLBIPQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:18.128263576 (564h2m38.620696025s) [noise=true , address=145.79.227.4:4187 , piece_id=DTUG63TSKA4F6PVSYTEW62Z466EU6JL3D7AIZ3S4XW26PE2QD2JA, seg_pos={0 0}, upload=496203]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op Sep 22 02:10:18.128321945 (564h2m38.620754377s) [noise=true , address=145.79.227.12:4765 , piece_id=7CVNMSIETGVFB2HGTSSED5D2K5H7TRBS2S26DQA2GJBJRMNEIEPQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:18.128331694 (564h2m38.620764139s) [noise=true , address=145.79.227.4:4186 , piece_id=WIQBBJOJSIBAZ6SR7YXFNOPM3EFE3SFJ3SN5TOVMNYWMPL5KFU3A, seg_pos={0 0}, upload=496203]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:18.128405121 (564h2m38.62083755s) [noise=true , address=145.79.227.14:4838 , piece_id=G33CKDP4E47EU5Q5PBV3YMYKALGDO5UKDGO53FV75FEKPLJMAWMQ, seg_pos={0 0}, upload=496203]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:10:18.128451190 (564h2m38.620883637s) [seg_pos={0 0}, upload=496203]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 22 02:10:18.234335390 (564h2m38.726767826s) [upload=496203]: Done uploading segments... Sep 22 02:10:18.234364755 (564h2m38.72679719s) [upload=496203]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.108490291s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.113374379s FAIL (2025-09-22 02:09:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 67s) [upload=496189]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:52.225736199 (564h1m12.718168646s) [upload=496189]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:52.451833105 (564h1m12.944265538s) [seg_pos={0 0}, upload=496189]: Begin upload segment... Sep 22 02:08:52.451902738 (564h1m12.944335174s) [seg_pos={0 0}, upload=496189]: Done begin upload segment. Sep 22 02:08:52.451939389 (564h1m12.944371841s) [noise=true , address=145.79.227.28:4752 , piece_id=2SXHZWKSSBY5NW355FYNDIHCVTAXVQ4XPHTK754DXOAG5CHXGBGA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.451950010 (564h1m12.944382463s) [noise=true , address=77.90.31.21:4894 , piece_id=XK7KV52KSYGENYOKCPKTBJSH5I43TY4CBEJKHISOTOAMWQVA7OAA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.452326618 (564h1m12.944759066s) [noise=true , address=145.79.227.4:4200 , piece_id=IT7TTQUMHMDOSATJQ2Z4YFK25DL7XTXEIPG5QO26NEIVY2STT2IA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.452343872 (564h1m12.944776318s) [noise=true , address=77.90.31.11:4579 , piece_id=URFCNEACJQLD43QNLFLJRVFZXFNAMEL5K6NBEEO2KRTXFI7D3DFA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.452637789 (564h1m12.945070236s) [noise=true , address=145.79.227.43:4191 , piece_id=2LHAQUCDQZZU6YM664LCGFWSTQMWXJBHRRJMKN6UDR4XIBICTKIA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.452936614 (564h1m12.945369055s) [noise=true , address=145.79.227.2:4142 , piece_id=3GNN7QBRABMP64MY2OCYKEFFVVCD4BJZ3RUXOYPCWJMJUBFMWA3A, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.453317750 (564h1m12.945750195s) [noise=true , address=77.90.31.16:4768 , piece_id=RMJZVRJ6VSOWEBTASIS6BCP3V5POLSHPHJD6LJP5BPKSX77NJ6RA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.453705789 (564h1m12.946138234s) [noise=true , address=145.79.227.29:4943 , piece_id=THG63XVRYR3JTFX2ZTMMODJTJGOBKDEKQOQXLCS7624HVEADYEOQ, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.458472802 (564h1m12.950905254s) [noise=true , address=145.79.227.8:4333 , piece_id=J2FEEWISBIQW7AYLRUDYZYXU4SZDYOT52LOJCIH4ELGCK7Q7ZAKA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.458912219 (564h1m12.951344671s) [noise=true , address=145.79.227.27:4692 , piece_id=NDCY4HOXDW2VIZZZW4Z5YQKTA2FXY6DER4KUHFZT7C3WJMV5KNAA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.459231228 (564h1m12.951663677s) [noise=true , address=145.79.227.24:4597 , piece_id=CBGDDFW2QJGICDDPTDGMOJQAJLIVXDWTHYA3MCHH4DEMR25B2Y2Q, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.459528774 (564h1m12.951961218s) [noise=true , address=145.79.227.33:4874 , piece_id=HEEZ6ZZ6T5BCEATM2K72FU2IBQA7GUKN6PVMQC4J7UHIDPFKS72A, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.459834664 (564h1m12.952267116s) [noise=true , address=145.79.227.16:4915 , piece_id=MDRWQMGWZWUAN4IYOVKOA42DMDHLCCMJHT7TVEN4ZISMYQBG7PDA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.460171789 (564h1m12.952604237s) [noise=true , address=77.90.31.19:4851 , piece_id=S5L2UWFCB5CX5EASMRM7BU4OJG6A7FSMMPIGY4QCTBTPFITFDJCQ, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.460436460 (564h1m12.952868904s) [noise=true , address=145.79.227.32:4863 , piece_id=46F2MMO2KOYSROQM56GBHDKH356VYDAIO26D2G57554X264IXXLQ, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.460677011 (564h1m12.953109458s) [noise=true , address=145.79.227.10:4429 , piece_id=IC5RHHTESBG62HAV6SJZVAKAL3GEDJR7H6OJTKOMZONUQ3N6PQMQ, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.460904563 (564h1m12.953337005s) [noise=true , address=145.79.227.5:4235 , piece_id=J7FUYWD32ZYNEBK3DKHW67HW6W2GWOWVXVBZEUTHRWIJBOGGZBOQ, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:52.462468855 (564h1m12.954901307s) [noise=true , address=145.79.227.51:4441 , piece_id=GU3JNR6SQZIWCB7FP365UEERNZGIQM6HIGTC3UD3D5BFG4DVFTXA, seg_pos={0 0}, upload=496189]: Uploading piece... Sep 22 02:08:53.025012884 (564h1m13.51744532s) [upload=496189]: Next returned nil segment Sep 22 02:08:53.025046523 (564h1m13.517478968s) [upload=496189]: Waiting for error group managing segments... Sep 22 02:08:53.165189593 (564h1m13.657622029s) [noise=true , address=77.90.31.16:4768 , piece_id=RMJZVRJ6VSOWEBTASIS6BCP3V5POLSHPHJD6LJP5BPKSX77NJ6RA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.165865720 (564h1m13.658298158s) [noise=true , address=77.90.31.19:4851 , piece_id=S5L2UWFCB5CX5EASMRM7BU4OJG6A7FSMMPIGY4QCTBTPFITFDJCQ, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.176174003 (564h1m13.668606439s) [noise=true , address=77.90.31.11:4579 , piece_id=URFCNEACJQLD43QNLFLJRVFZXFNAMEL5K6NBEEO2KRTXFI7D3DFA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.213469356 (564h1m13.705901805s) [noise=true , address=145.79.227.32:4863 , piece_id=46F2MMO2KOYSROQM56GBHDKH356VYDAIO26D2G57554X264IXXLQ, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.220674622 (564h1m13.713107087s) [noise=true , address=145.79.227.28:4752 , piece_id=2SXHZWKSSBY5NW355FYNDIHCVTAXVQ4XPHTK754DXOAG5CHXGBGA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.223405708 (564h1m13.715838155s) [noise=true , address=145.79.227.43:4191 , piece_id=2LHAQUCDQZZU6YM664LCGFWSTQMWXJBHRRJMKN6UDR4XIBICTKIA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.230589696 (564h1m13.723022142s) [noise=true , address=145.79.227.24:4597 , piece_id=CBGDDFW2QJGICDDPTDGMOJQAJLIVXDWTHYA3MCHH4DEMR25B2Y2Q, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.230879694 (564h1m13.72331213s) [noise=true , address=145.79.227.33:4874 , piece_id=HEEZ6ZZ6T5BCEATM2K72FU2IBQA7GUKN6PVMQC4J7UHIDPFKS72A, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.233445822 (564h1m13.725878272s) [noise=true , address=145.79.227.51:4441 , piece_id=GU3JNR6SQZIWCB7FP365UEERNZGIQM6HIGTC3UD3D5BFG4DVFTXA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.236514045 (564h1m13.728946498s) [noise=true , address=145.79.227.27:4692 , piece_id=NDCY4HOXDW2VIZZZW4Z5YQKTA2FXY6DER4KUHFZT7C3WJMV5KNAA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.333050979 (564h1m13.825483413s) [noise=true , address=77.90.31.21:4894 , piece_id=XK7KV52KSYGENYOKCPKTBJSH5I43TY4CBEJKHISOTOAMWQVA7OAA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:08:53.412042029 (564h1m13.904474475s) [noise=true , address=145.79.227.29:4943 , piece_id=THG63XVRYR3JTFX2ZTMMODJTJGOBKDEKQOQXLCS7624HVEADYEOQ, seg_pos={0 0}, upload=496189]: Done uploading piece. err: <nil> Sep 22 02:09:51.917230049 (564h2m12.409662492s) [noise=true , address=145.79.227.5:4235 , piece_id=J7FUYWD32ZYNEBK3DKHW67HW6W2GWOWVXVBZEUTHRWIJBOGGZBOQ, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:1nn8EX7mq6mXic78Jura9BcAHfgw8rFwzfqf5HtwJtrYt6PUDV, address:145.79.227.5:4235): protocol: rpc: dial tcp 145.79.227.5:4235: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.917421848 (564h2m12.409854292s) [noise=true , address=145.79.227.10:4429 , piece_id=IC5RHHTESBG62HAV6SJZVAKAL3GEDJR7H6OJTKOMZONUQ3N6PQMQ, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:1Bi1qafYFw9NukFHUJzV9M4Jd1i7YsjQMs7jXsVUn44pecVWzr, address:145.79.227.10:4429): protocol: rpc: dial tcp 145.79.227.10:4429: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.917496978 (564h2m12.409929416s) [noise=true , address=145.79.227.16:4915 , piece_id=MDRWQMGWZWUAN4IYOVKOA42DMDHLCCMJHT7TVEN4ZISMYQBG7PDA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: rpc: dial tcp 145.79.227.16:4915: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.917722143 (564h2m12.410154589s) [noise=true , address=145.79.227.8:4333 , piece_id=J2FEEWISBIQW7AYLRUDYZYXU4SZDYOT52LOJCIH4ELGCK7Q7ZAKA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:145.79.227.8:4333): protocol: rpc: dial tcp 145.79.227.8:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.917960814 (564h2m12.410393266s) [noise=true , address=145.79.227.4:4200 , piece_id=IT7TTQUMHMDOSATJQ2Z4YFK25DL7XTXEIPG5QO26NEIVY2STT2IA, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:12dfLumooVsDsWXqi5FQMDutkcexM9A5Kmu4Vo9s1TqfWxv5ozH, address:145.79.227.4:4200): protocol: rpc: dial tcp 145.79.227.4:4200: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.917993601 (564h2m12.410426032s) [noise=true , address=145.79.227.2:4142 , piece_id=3GNN7QBRABMP64MY2OCYKEFFVVCD4BJZ3RUXOYPCWJMJUBFMWA3A, seg_pos={0 0}, upload=496189]: Done uploading piece. err: ecclient: upload failed (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7, address:145.79.227.2:4142): protocol: rpc: dial tcp 145.79.227.2:4142: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:51.918047425 (564h2m12.410479878s) [seg_pos={0 0}, upload=496189]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 02:09:52.028488207 (564h2m12.520920642s) [upload=496189]: Done uploading segments... Sep 22 02:09:52.028517254 (564h2m12.520949685s) [upload=496189]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.113438203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.124604755s FAIL (2025-09-22 02:09:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 02:08:49.550261963 (564h1m10.042694395s) [upload=496188]: Starting upload Sep 22 02:08:49.550335443 (564h1m10.042767885s) [upload=496188]: Done starting upload Sep 22 02:08:49.550471129 (564h1m10.042903594s) [upload=496188]: Uploading segments... Sep 22 02:08:49.550661656 (564h1m10.043094099s) [upload=496188]: Got next segment. Inline: false Sep 22 02:08:49.550702450 (564h1m10.043134884s) [upload=496188]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:08:49.550707896 (564h1m10.043140328s) [upload=496188]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:08:49.698734043 (564h1m10.191166494s) [seg_pos={0 0}, upload=496188]: Begin upload segment... Sep 22 02:08:49.698807151 (564h1m10.19123958s) [seg_pos={0 0}, upload=496188]: Done begin upload segment. Sep 22 02:08:49.698841070 (564h1m10.191273522s) [noise=true , address=145.79.227.14:4840 , piece_id=VSUSOV7LG4YA5MDPXINCWNA5B43I57UHKBQ7D6AGTQBW2LPEFTKQ, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.698994643 (564h1m10.191427107s) [noise=true , address=45.91.116.6:4973 , piece_id=QNXX4ZNPE4PA4BLSHWNBU24YYAQ2FOQJZXCJVY3DJSUYOIHFB6FA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.699287324 (564h1m10.191719774s) [noise=true , address=145.79.227.15:4876 , piece_id=OLK72NPN6KCYHYGKVRZ4GBMQPDOFETY7SPDQ4JLPUCOFBUHZWWZA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.699386341 (564h1m10.191818783s) [noise=true , address=145.79.227.13:4802 , piece_id=MHKEWXRMRC5MKIXVSSWUDFPGON36ZL2VYEG3JOOGUWCHDNJQTNJQ, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.699675559 (564h1m10.192108002s) [noise=true , address=77.90.31.21:4877 , piece_id=4R6CNH75KJTK3PEGK2KBLFH7M2AUZWOMUIPJHDWLIUL4L3IKNR6Q, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.699704661 (564h1m10.192137115s) [noise=true , address=145.79.227.11:4731 , piece_id=ZI6MSJP2OFURUNXLGXJRDW3YU2JMC72R5BYNJWPJAPUU2QEECUZA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.699973896 (564h1m10.192406338s) [noise=true , address=45.91.116.9:4262 , piece_id=QRJSMVGA6VRXYB5A34DLKZD42XQE6U5AJ5UUAXEGPG5NCRURZXUA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.700876483 (564h1m10.193308926s) [noise=true , address=77.90.31.16:4735 , piece_id=53U35QJY2DKMP4PCHZNNQ26DB7VW23BHG3JD2BFAV63QE23SF56A, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.700944384 (564h1m10.193376837s) [noise=true , address=145.79.227.17:4970 , piece_id=WFPSCHSIIKXH4LNUE7UX7Z23TCIPNOLTN7X4L4NODVMDXOY4OXZA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.701319403 (564h1m10.193751861s) [noise=true , address=145.79.227.3:4149 , piece_id=YWFS2IYTFLEORUCGCYDJCYAC4XDVR7CEI4UGEM7U3ODVDHTDKT6A, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.701761024 (564h1m10.194193472s) [noise=true , address=45.91.116.12:4226 , piece_id=XVE323ND6U3A2BDNFXNV3S4ADLMW6PKEJJQVYANDIQAJO3CNWIXA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.702566921 (564h1m10.194999373s) [noise=true , address=145.79.227.14:4837 , piece_id=2GIY3Z54K2BPKX7OLVFYTOBN7CH7M6ORLLRCBX6OQJOAWQIIBZAA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.702965226 (564h1m10.195397685s) [noise=true , address=45.91.116.13:4153 , piece_id=UWRLHLNJ3UDTSJIBSWLE6D4K2AMCY342E2TIJGFEGHUV7CTNUX6A, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.703615455 (564h1m10.196047907s) [noise=true , address=45.91.116.12:4227 , piece_id=OMZOJETZS4PMHKNTWJU3PM5K3E6F3ZFBQRIP4HHIR5PAKLTBP5VQ, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.704104323 (564h1m10.196536775s) [noise=true , address=45.91.116.10:4081 , piece_id=NUFBJSMEQ47LEVKZ7AU5R63E7T7KNYHTD6WRI7LG5IRMGKUE2QYA, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:49.705336443 (564h1m10.197768898s) [noise=true , address=45.91.116.11:4117 , piece_id=HSODDHYV6P27NGPNHUSURGA32IJTIOMG2XN3QXI6NKMSMZQIWR3Q, seg_pos={0 0}, upload=496188]: Uploading piece... Sep 22 02:08:50.319794766 (564h1m10.8122272s) [upload=496188]: Next returned nil segment Sep 22 02:08:50.319813724 (564h1m10.812246157s) [upload=496188]: Waiting for error group managing segments... Sep 22 02:08:50.457042588 (564h1m10.949475031s) [noise=true , address=45.91.116.12:4226 , piece_id=XVE323ND6U3A2BDNFXNV3S4ADLMW6PKEJJQVYANDIQAJO3CNWIXA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.457434439 (564h1m10.949866874s) [noise=true , address=77.90.31.21:4877 , piece_id=4R6CNH75KJTK3PEGK2KBLFH7M2AUZWOMUIPJHDWLIUL4L3IKNR6Q, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.467412640 (564h1m10.959845069s) [noise=true , address=45.91.116.13:4153 , piece_id=UWRLHLNJ3UDTSJIBSWLE6D4K2AMCY342E2TIJGFEGHUV7CTNUX6A, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.470304073 (564h1m10.962736521s) [noise=true , address=45.91.116.10:4081 , piece_id=NUFBJSMEQ47LEVKZ7AU5R63E7T7KNYHTD6WRI7LG5IRMGKUE2QYA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.470712648 (564h1m10.963145083s) [noise=true , address=45.91.116.6:4973 , piece_id=QNXX4ZNPE4PA4BLSHWNBU24YYAQ2FOQJZXCJVY3DJSUYOIHFB6FA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.475962109 (564h1m10.968394545s) [noise=true , address=45.91.116.9:4262 , piece_id=QRJSMVGA6VRXYB5A34DLKZD42XQE6U5AJ5UUAXEGPG5NCRURZXUA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.475997344 (564h1m10.96842978s) [noise=true , address=45.91.116.12:4227 , piece_id=OMZOJETZS4PMHKNTWJU3PM5K3E6F3ZFBQRIP4HHIR5PAKLTBP5VQ, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.479058846 (564h1m10.971491288s) [noise=true , address=77.90.31.16:4735 , piece_id=53U35QJY2DKMP4PCHZNNQ26DB7VW23BHG3JD2BFAV63QE23SF56A, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:08:50.479584396 (564h1m10.972016824s) [noise=true , address=45.91.116.11:4117 , piece_id=HSODDHYV6P27NGPNHUSURGA32IJTIOMG2XN3QXI6NKMSMZQIWR3Q, seg_pos={0 0}, upload=496188]: Done uploading piece. err: <nil> Sep 22 02:09:19.245552311 (564h1m39.737984742s) [noise=true , address=145.79.227.14:4837 , piece_id=2GIY3Z54K2BPKX7OLVFYTOBN7CH7M6ORLLRCBX6OQJOAWQIIBZAA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: rpc: dial tcp 145.79.227.14:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:19.245648479 (564h1m39.738080922s) [noise=true , address=145.79.227.3:4149 , piece_id=YWFS2IYTFLEORUCGCYDJCYAC4XDVR7CEI4UGEM7U3ODVDHTDKT6A, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: rpc: dial tcp 145.79.227.3:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:19.245750429 (564h1m39.738182873s) [noise=true , address=145.79.227.14:4840 , piece_id=VSUSOV7LG4YA5MDPXINCWNA5B43I57UHKBQ7D6AGTQBW2LPEFTKQ, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:19.245771853 (564h1m39.738204302s) [noise=true , address=145.79.227.17:4970 , piece_id=WFPSCHSIIKXH4LNUE7UX7Z23TCIPNOLTN7X4L4NODVMDXOY4OXZA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:19.245856706 (564h1m39.738289149s) [noise=true , address=145.79.227.15:4876 , piece_id=OLK72NPN6KCYHYGKVRZ4GBMQPDOFETY7SPDQ4JLPUCOFBUHZWWZA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: rpc: dial tcp 145.79.227.15:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:09:19.245884256 (564h1m39.738316701s) [noise=true , address=145.79.227.11:4731 , piece_id=ZI6MSJP2OFURUNXLGXJRDW3YU2JMC72R5BYNJWPJAPUU2QEECUZA, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:09:19.246094328 (564h1m39.738526772s) [noise=true , address=145.79.227.13:4802 , piece_id=MHKEWXRMRC5MKIXVSSWUDFPGON36ZL2VYEG3JOOGUWCHDNJQTNJQ, seg_pos={0 0}, upload=496188]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:09:19.246220007 (564h1m39.738652459s) [seg_pos={0 0}, upload=496188]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 02:09:19.368116906 (564h1m39.860549338s) [upload=496188]: Done uploading segments... Sep 22 02:09:19.368167900 (564h1m39.860600346s) [upload=496188]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.124648443s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.111294269s FAIL (2025-09-22 02:08:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 50125s) [seg_pos={0 0}, upload=496171]: Done begin upload segment. Sep 22 02:07:52.151102539 (564h0m12.643534992s) [noise=true , address=145.79.227.23:4563 , piece_id=HURQTMENU5B75OTCSBYWT54R2I37KPPIEXFLTCTF6YDSAC3O3CBA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.151719798 (564h0m12.644152249s) [noise=true , address=145.79.227.19:4081 , piece_id=OEFOIZI7IGPXZSTVCB6DTR55TA3VYJ7MSIFGXTURYOPOMS5P62MQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.152987578 (564h0m12.645420029s) [noise=true , address=145.79.227.32:4835 , piece_id=XTA5ADR3EHDLPKEDE7YIBMAQHAQGELYT44C3D6AHOKN4G2QPR5HA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.153329465 (564h0m12.645761907s) [noise=true , address=145.79.227.51:4417 , piece_id=P7LOE3XCW3GU25NN7ECHT7GQRBQAXYOFLKENDNH5B5I2FI7SQ4IA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.153761208 (564h0m12.646193653s) [noise=true , address=145.79.227.10:4407 , piece_id=WZNTRRW6XV3TSVJJM4GSBD6RVQZ4ICW3CVXVGORXS7CAPDFCY2EQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.153984054 (564h0m12.646416501s) [noise=true , address=145.79.227.7:4322 , piece_id=PPXL4SI64YUU6JGYNTVPXIDC5WF46GLM3PDKAE7BUBSPRWN42UBA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.154696930 (564h0m12.647129375s) [noise=true , address=145.79.227.11:4734 , piece_id=COM4MTEQGKUVLFCATMVAAJI2UKA7PDKQONIYH4OMSQAT54FCONEA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.154814781 (564h0m12.647247223s) [noise=true , address=145.79.227.15:4906 , piece_id=KL3R7ION2WSIJIWEDBAGOR7GWMG37SBR5RF674W7KO3NBUWRMBPQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.154874507 (564h0m12.647306949s) [noise=true , address=145.79.227.43:4171 , piece_id=MA64YZKBI56D7ZW5GKDFG3EO4GHYB7NJF5ECDCKWQAFATYE2AT2A, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.154932909 (564h0m12.647365355s) [noise=true , address=145.79.227.48:4310 , piece_id=PAOHYEX4ZFTPE55Q2TGKWNWARRQAAYELGWH32A5ZJ6GF3KMQU7LA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.155013941 (564h0m12.647446386s) [noise=true , address=145.79.227.45:4238 , piece_id=KBVNAH5HGKIUBU33HSDCP6XHH5E4FAEJDGCMOYKCRG5QPG4DKDFQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.155081236 (564h0m12.647513684s) [noise=true , address=77.90.31.21:4901 , piece_id=B3GNU2PAEIK6GZNTO6RI4PGAKFDWGCRWVNJ5RWBQGWUC5TSCAGAA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.155434982 (564h0m12.64786743s) [noise=true , address=145.79.227.39:4015 , piece_id=DPWU6QV6NPT43RYPRDZJWJUUOMK4V5Q7HC3IKS3HEOND75VQ723A, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.155728477 (564h0m12.648160926s) [noise=true , address=145.79.227.16:4916 , piece_id=UQD5MZTBOJEE55IV5F32HE7FBOCUO6PX26YJ67IYWIFYYQ3SO2KQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.157402036 (564h0m12.649834487s) [noise=true , address=145.79.227.11:4754 , piece_id=7S2IR3D3JC6UTOZWKE2MDPH5LAOKB5IFIA3FM27YZJJPYKXWB6OQ, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.157726989 (564h0m12.65015944s) [noise=true , address=145.79.227.4:4193 , piece_id=TLMDJIISTLQO4JPJGTKQE76SV5IVP5LAVOUYM3HH7Q6MFFGN4ITA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.158169452 (564h0m12.650601906s) [noise=true , address=145.79.227.2:4116 , piece_id=3EMHEW6VVYOUYT2VXTRDPHAE6FE7H35HCI6ML3MNH6P4AWP53VKA, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.158438722 (564h0m12.650871167s) [noise=true , address=145.79.227.25:4650 , piece_id=ZXQKR2ENO7WSE44VFCD7OUHU7IZ7YZHR2ZP6XD2AFEMINRFQRR3A, seg_pos={0 0}, upload=496171]: Uploading piece... Sep 22 02:07:52.741631087 (564h0m13.23406352s) [upload=496171]: Next returned nil segment Sep 22 02:07:52.741656668 (564h0m13.23408912s) [upload=496171]: Waiting for error group managing segments... Sep 22 02:07:52.886953369 (564h0m13.379385822s) [noise=true , address=77.90.31.21:4901 , piece_id=B3GNU2PAEIK6GZNTO6RI4PGAKFDWGCRWVNJ5RWBQGWUC5TSCAGAA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.911903433 (564h0m13.40433588s) [noise=true , address=145.79.227.32:4835 , piece_id=XTA5ADR3EHDLPKEDE7YIBMAQHAQGELYT44C3D6AHOKN4G2QPR5HA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.922694541 (564h0m13.415126983s) [noise=true , address=145.79.227.19:4081 , piece_id=OEFOIZI7IGPXZSTVCB6DTR55TA3VYJ7MSIFGXTURYOPOMS5P62MQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.922944015 (564h0m13.415376467s) [noise=true , address=145.79.227.25:4650 , piece_id=ZXQKR2ENO7WSE44VFCD7OUHU7IZ7YZHR2ZP6XD2AFEMINRFQRR3A, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.925392089 (564h0m13.417824539s) [noise=true , address=145.79.227.43:4171 , piece_id=MA64YZKBI56D7ZW5GKDFG3EO4GHYB7NJF5ECDCKWQAFATYE2AT2A, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.928362902 (564h0m13.420795354s) [noise=true , address=145.79.227.45:4238 , piece_id=KBVNAH5HGKIUBU33HSDCP6XHH5E4FAEJDGCMOYKCRG5QPG4DKDFQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.928489543 (564h0m13.42092198s) [noise=true , address=145.79.227.23:4563 , piece_id=HURQTMENU5B75OTCSBYWT54R2I37KPPIEXFLTCTF6YDSAC3O3CBA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.940929273 (564h0m13.433361708s) [noise=true , address=145.79.227.39:4015 , piece_id=DPWU6QV6NPT43RYPRDZJWJUUOMK4V5Q7HC3IKS3HEOND75VQ723A, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.944387421 (564h0m13.436819858s) [noise=true , address=145.79.227.51:4417 , piece_id=P7LOE3XCW3GU25NN7ECHT7GQRBQAXYOFLKENDNH5B5I2FI7SQ4IA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:07:52.953779671 (564h0m13.446212107s) [noise=true , address=145.79.227.48:4310 , piece_id=PAOHYEX4ZFTPE55Q2TGKWNWARRQAAYELGWH32A5ZJ6GF3KMQU7LA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: <nil> Sep 22 02:08:51.639210898 (564h1m12.131643332s) [noise=true , address=145.79.227.15:4906 , piece_id=KL3R7ION2WSIJIWEDBAGOR7GWMG37SBR5RF674W7KO3NBUWRMBPQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:1gByS6jTUkLuHrY8MafgmkmB4wXyY2j9Mt48fkpPNWbCKTcnty, address:145.79.227.15:4906): protocol: rpc: dial tcp 145.79.227.15:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639292109 (564h1m12.131724552s) [noise=true , address=145.79.227.11:4754 , piece_id=7S2IR3D3JC6UTOZWKE2MDPH5LAOKB5IFIA3FM27YZJJPYKXWB6OQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:125aPwoHeFabk6zZ6bkGNj3USJYgTQG3RB1QhhXssj7Vik2AuCs, address:145.79.227.11:4754): protocol: rpc: dial tcp 145.79.227.11:4754: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639361872 (564h1m12.131794307s) [noise=true , address=145.79.227.7:4322 , piece_id=PPXL4SI64YUU6JGYNTVPXIDC5WF46GLM3PDKAE7BUBSPRWN42UBA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:12WwYqhEYC9QPkGDiSt5bF7izvhpsawWx5dwTqP5zZcPyigs1n3, address:145.79.227.7:4322): protocol: rpc: dial tcp 145.79.227.7:4322: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639410938 (564h1m12.131843382s) [noise=true , address=145.79.227.16:4916 , piece_id=UQD5MZTBOJEE55IV5F32HE7FBOCUO6PX26YJ67IYWIFYYQ3SO2KQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:12fifEPABGWmVzxBHNzhjh3r4ue3RRSeNdydifpXruXtH4FELBK, address:145.79.227.16:4916): protocol: rpc: dial tcp 145.79.227.16:4916: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639472148 (564h1m12.131904582s) [noise=true , address=145.79.227.4:4193 , piece_id=TLMDJIISTLQO4JPJGTKQE76SV5IVP5LAVOUYM3HH7Q6MFFGN4ITA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:15XNLgptmXprcfkVMJsuzY2WDyHiZDpLetiBpmjXhb4c9q7yMd, address:145.79.227.4:4193): protocol: rpc: dial tcp 145.79.227.4:4193: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639531573 (564h1m12.131964014s) [noise=true , address=145.79.227.2:4116 , piece_id=3EMHEW6VVYOUYT2VXTRDPHAE6FE7H35HCI6ML3MNH6P4AWP53VKA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:123aoo9g11aJP9hiL8shu2dx44tjyKzEWqPUjHUdXtR7LLFyng8, address:145.79.227.2:4116): protocol: rpc: dial tcp 145.79.227.2:4116: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:51.639636008 (564h1m12.132068441s) [noise=true , address=145.79.227.10:4407 , piece_id=WZNTRRW6XV3TSVJJM4GSBD6RVQZ4ICW3CVXVGORXS7CAPDFCY2EQ, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:1kJKjEfzwg1wuBUP2Bw4oaSoy6mQZdvDED6PpkbrqQwbkh3bUW, address:145.79.227.10:4407): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:51.639836825 (564h1m12.13226927s) [noise=true , address=145.79.227.11:4734 , piece_id=COM4MTEQGKUVLFCATMVAAJI2UKA7PDKQONIYH4OMSQAT54FCONEA, seg_pos={0 0}, upload=496171]: Done uploading piece. err: ecclient: upload failed (node:12GLb5LQje9wwivnj8Wj1tXcnS6mFb2Bco5saNwgmFJBVDujrv7, address:145.79.227.11:4734): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:51.639869605 (564h1m12.132302058s) [seg_pos={0 0}, upload=496171]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Sep 22 02:08:51.747186945 (564h1m12.239619381s) [upload=496171]: Done uploading segments... Sep 22 02:08:51.747213805 (564h1m12.23964624s) [upload=496171]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.111314738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.114125266s FAIL (2025-09-22 02:08:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:07:52.471101177 (564h0m12.963533621s) [upload=496172]: Starting upload Sep 22 02:07:52.471240294 (564h0m12.963672726s) [upload=496172]: Done starting upload Sep 22 02:07:52.471483895 (564h0m12.963916343s) [upload=496172]: Uploading segments... Sep 22 02:07:52.471697939 (564h0m12.964130375s) [upload=496172]: Got next segment. Inline: false Sep 22 02:07:52.471769975 (564h0m12.964202419s) [upload=496172]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:07:52.471777003 (564h0m12.964209451s) [upload=496172]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:07:52.592111331 (564h0m13.084543773s) [seg_pos={0 0}, upload=496172]: Begin upload segment... Sep 22 02:07:52.592181955 (564h0m13.084614406s) [seg_pos={0 0}, upload=496172]: Done begin upload segment. Sep 22 02:07:52.592213421 (564h0m13.084645869s) [noise=true , address=77.90.31.12:4592 , piece_id=F3F6ZIMQMQWCMMJM7XW72EPUMDZ2JYOTYYXTBGMTJO7L7VCF5RVQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.592237038 (564h0m13.084669481s) [noise=true , address=77.90.31.15:4699 , piece_id=WC4T7JYEQE7252XUBCWCJLT5LJTKN3PH3W72CAEZALUPMVVD3RFA, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.592687316 (564h0m13.085119765s) [noise=true , address=45.91.116.9:4261 , piece_id=NTZL7QQFZY6VSPDXO73K4VZI4GSBHPC26DLNICWIPSTG2DV4WN4Q, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.592722305 (564h0m13.085154756s) [noise=true , address=145.79.227.11:4729 , piece_id=E7DCLMDHLPKNAOSG5JLHRFVGO3OXVLK2WTUUTA3EDHNT7UDFVUFA, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593083886 (564h0m13.085516328s) [noise=true , address=145.79.227.12:4765 , piece_id=WTRT7VXQCWGQKZHLJVFVY7UTD3SOIZPCBK5BUDAAQ6DIYNY5APQQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593121241 (564h0m13.085553689s) [noise=true , address=77.90.31.10:4518 , piece_id=RT7ME5DEMFMVNO56X2AFM2JF2K5TNBOXU64AZCTXVKEGEZ5FOIPQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593417546 (564h0m13.085849991s) [noise=true , address=145.79.227.11:4730 , piece_id=ZKR6MVG535AO224POSKQ6QRYRMW2J6ONFCR2I2PEC6ZJVLM4NJAA, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593513058 (564h0m13.085945503s) [noise=true , address=77.90.31.17:4772 , piece_id=C5Z4WQSX2BYO66WOQ7MSAMX7VYGJPVNXFCJUS2MMPSVER7EPT4QQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593729843 (564h0m13.086162285s) [noise=true , address=77.90.31.14:4663 , piece_id=DDW5NYZOWT7FFPK4ADK5BRYL6WG3ZARXUV5Z7QW4VCJ5FNSGX6JQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.593977938 (564h0m13.086410392s) [noise=true , address=77.90.31.16:4734 , piece_id=5U7FVPHEIUHKBJ55KSLHF3MKMSOA3FNAXU573WDMC4CMQBRFLY3Q, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.594061000 (564h0m13.086493446s) [noise=true , address=45.91.116.6:4973 , piece_id=6Z2VGKJ7BXHMSZXEFS2PEXOOX76BW3H2SKBV6LZ2EOA3HHQTTZTQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.594490895 (564h0m13.086923346s) [noise=true , address=45.91.116.10:4082 , piece_id=NCGQFW37YB2NUUMIYHJAII6UGGKJLQXHXJEDBZJAHCPYMWZIVPOQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.594803647 (564h0m13.087236093s) [noise=true , address=77.90.31.16:4735 , piece_id=VVGHE5HZVXSCDG6R267PH63UJD5VUB5KAOWT22SE5Z3OWEFNKG4A, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.595201618 (564h0m13.087634053s) [noise=true , address=77.90.31.19:4843 , piece_id=R226ZMQDATZ7DME6LV2RRAF4VPVWBZMUEMJZDWEOSUZRTW45BMSA, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.596784514 (564h0m13.089216955s) [noise=true , address=145.79.227.7:4294 , piece_id=AJ47ED64UQKPKH5YEWVYGWZHGURNWEBCN7KP4QSMMAIFYQQE2DSQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:52.597130664 (564h0m13.089563099s) [noise=true , address=45.91.116.14:4191 , piece_id=7UUQODROQ34UJFYYZNKCF6EKIX34KDR2LMWV23PK34ZHIBZQRVXQ, seg_pos={0 0}, upload=496172]: Uploading piece... Sep 22 02:07:53.208365628 (564h0m13.700798057s) [upload=496172]: Next returned nil segment Sep 22 02:07:53.208384267 (564h0m13.700816713s) [upload=496172]: Waiting for error group managing segments... Sep 22 02:07:53.342602002 (564h0m13.835034449s) [noise=true , address=45.91.116.10:4082 , piece_id=NCGQFW37YB2NUUMIYHJAII6UGGKJLQXHXJEDBZJAHCPYMWZIVPOQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.343496799 (564h0m13.835929231s) [noise=true , address=45.91.116.14:4191 , piece_id=7UUQODROQ34UJFYYZNKCF6EKIX34KDR2LMWV23PK34ZHIBZQRVXQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.343599374 (564h0m13.836031802s) [noise=true , address=77.90.31.12:4592 , piece_id=F3F6ZIMQMQWCMMJM7XW72EPUMDZ2JYOTYYXTBGMTJO7L7VCF5RVQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.344707481 (564h0m13.837139917s) [noise=true , address=45.91.116.9:4261 , piece_id=NTZL7QQFZY6VSPDXO73K4VZI4GSBHPC26DLNICWIPSTG2DV4WN4Q, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.345292434 (564h0m13.837724868s) [noise=true , address=77.90.31.15:4699 , piece_id=WC4T7JYEQE7252XUBCWCJLT5LJTKN3PH3W72CAEZALUPMVVD3RFA, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.345828534 (564h0m13.83826098s) [noise=true , address=77.90.31.16:4735 , piece_id=VVGHE5HZVXSCDG6R267PH63UJD5VUB5KAOWT22SE5Z3OWEFNKG4A, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.345902318 (564h0m13.838334751s) [noise=true , address=45.91.116.6:4973 , piece_id=6Z2VGKJ7BXHMSZXEFS2PEXOOX76BW3H2SKBV6LZ2EOA3HHQTTZTQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.350296151 (564h0m13.842728608s) [noise=true , address=77.90.31.19:4843 , piece_id=R226ZMQDATZ7DME6LV2RRAF4VPVWBZMUEMJZDWEOSUZRTW45BMSA, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.356003609 (564h0m13.848436037s) [noise=true , address=77.90.31.14:4663 , piece_id=DDW5NYZOWT7FFPK4ADK5BRYL6WG3ZARXUV5Z7QW4VCJ5FNSGX6JQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.356817581 (564h0m13.849250017s) [noise=true , address=77.90.31.16:4734 , piece_id=5U7FVPHEIUHKBJ55KSLHF3MKMSOA3FNAXU573WDMC4CMQBRFLY3Q, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.363040736 (564h0m13.855473165s) [noise=true , address=77.90.31.10:4518 , piece_id=RT7ME5DEMFMVNO56X2AFM2JF2K5TNBOXU64AZCTXVKEGEZ5FOIPQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:07:53.363908245 (564h0m13.856340696s) [noise=true , address=77.90.31.17:4772 , piece_id=C5Z4WQSX2BYO66WOQ7MSAMX7VYGJPVNXFCJUS2MMPSVER7EPT4QQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: <nil> Sep 22 02:08:22.153724640 (564h0m42.646157071s) [noise=true , address=145.79.227.7:4294 , piece_id=AJ47ED64UQKPKH5YEWVYGWZHGURNWEBCN7KP4QSMMAIFYQQE2DSQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:22.153861389 (564h0m42.646293834s) [noise=true , address=145.79.227.11:4729 , piece_id=E7DCLMDHLPKNAOSG5JLHRFVGO3OXVLK2WTUUTA3EDHNT7UDFVUFA, seg_pos={0 0}, upload=496172]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:08:22.153990451 (564h0m42.646422883s) [noise=true , address=145.79.227.12:4765 , piece_id=WTRT7VXQCWGQKZHLJVFVY7UTD3SOIZPCBK5BUDAAQ6DIYNY5APQQ, seg_pos={0 0}, upload=496172]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:22.154078128 (564h0m42.646510566s) [noise=true , address=145.79.227.11:4730 , piece_id=ZKR6MVG535AO224POSKQ6QRYRMW2J6ONFCR2I2PEC6ZJVLM4NJAA, seg_pos={0 0}, upload=496172]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:08:22.154248545 (564h0m42.646680993s) [seg_pos={0 0}, upload=496172]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:08:22.264725743 (564h0m42.757158198s) [upload=496172]: Done uploading segments... Sep 22 02:08:22.264759047 (564h0m42.7571915s) [upload=496172]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.11419677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.109354105s FAIL (2025-09-22 02:07:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== tarting upload Sep 22 02:06:51.711705129 (563h59m12.204137564s) [upload=496159]: Uploading segments... Sep 22 02:06:51.711753491 (563h59m12.204185922s) [upload=496159]: Got next segment. Inline: false Sep 22 02:06:51.711794898 (563h59m12.204227332s) [upload=496159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:06:51.711801188 (563h59m12.204233621s) [upload=496159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:06:51.935000632 (563h59m12.42743308s) [seg_pos={0 0}, upload=496159]: Begin upload segment... Sep 22 02:06:51.935130704 (563h59m12.427563151s) [noise=true , address=77.90.31.15:4725 , piece_id=QTTWTKGN3AD56XPDK5VTYVMEXCEPPJIOBPVMFHECMSMHJR2CWIHQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.935149201 (563h59m12.427581629s) [seg_pos={0 0}, upload=496159]: Done begin upload segment. Sep 22 02:06:51.935186203 (563h59m12.427618657s) [noise=true , address=145.79.227.24:4613 , piece_id=6XBZYZWLPG3NBGDDENT26MK5NJNU2R7BDZAZX5TUM4KIVD2KSQGQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.935702329 (563h59m12.428134778s) [noise=true , address=145.79.227.15:4885 , piece_id=57TWC6FA4VFH3XE7GNTOVLBQZKXCP4LSG6MZRZTDYZOBO3CLJIJA, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936020408 (563h59m12.428452853s) [noise=true , address=145.79.227.4:4217 , piece_id=BU7KXBLIYRNOMYFC4O3DWDU3FACZDOZZRDTWNS5JUZPBPXVHOLSQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936213602 (563h59m12.428646052s) [noise=true , address=145.79.227.16:4930 , piece_id=S5NCQZ5XUI5J4PNJXHT4YEPMBARGEQQCXUOFAX4WLMTBRU4PJR5A, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936390897 (563h59m12.428823339s) [noise=true , address=145.79.227.18:4014 , piece_id=7FJTCF52QTZHS2N43YTUPL2FXB654LZ5QM2VHDGNOMMZCE4B3KQQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936574458 (563h59m12.429006903s) [noise=true , address=145.79.227.10:4423 , piece_id=WQFRSEXMRVW7HNUMRJRU5TYCRIQVIB4HDNMZ4WGZOWMSHLMQEF2Q, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936662567 (563h59m12.429095012s) [noise=true , address=145.79.227.6:4261 , piece_id=SOKPD3KG2K4BCAS3LEZJ6BSFRWXW3INXXUC3D3CHU2B7IV5FQD3Q, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.936890879 (563h59m12.429323332s) [noise=true , address=145.79.227.29:4954 , piece_id=HIQWP3ERZH2XUEY2OI3WUJVWN2CVZBA3IEWKPFT5SE6BW3QBISWA, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.937254832 (563h59m12.429687283s) [noise=true , address=145.79.227.19:4111 , piece_id=FLUQF6UKUJUW3JAFK7EE4EBJ4XNEPLITIS3OKDNPXNDMNOZ5A65Q, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.937579979 (563h59m12.43001243s) [noise=true , address=64.49.13.169:4714 , piece_id=C27ZR5UAFL6N4R432UBHZPMDWJH3MILPCJRVX7SJFEZ3AZT37KRQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.938149159 (563h59m12.430581604s) [noise=true , address=145.79.227.13:4809 , piece_id=YCSFYNKMSP4SNBKUPGZS77UMUAVFKZKEHCZSGKDZM3MLIYDVJO4A, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.938695420 (563h59m12.431127862s) [noise=true , address=145.79.227.27:4712 , piece_id=G5S5CFXX244RWRR6QUPBWC6VTNWG3QPNWZVZMAVQQF7TVGNSRTGQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.939121175 (563h59m12.431553617s) [noise=true , address=145.79.227.39:4031 , piece_id=EQFA6RH43ZMQXUYUUTFNZLN5UPRHH7I7BKSQDYO7AQL2IMKANGJQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.939205759 (563h59m12.431638211s) [noise=true , address=145.79.227.2:4131 , piece_id=SUETKESYNBCHEKOXBHLHFRIZFSMFJPLEBFTM4GRPDUO6YE432JTA, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.939831937 (563h59m12.43226439s) [noise=true , address=145.79.227.40:4052 , piece_id=RPKJRMIVXJHRDPV4AECXF7SIMB7PJ7NDDR5M655UKM7EGESK44CA, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.940201242 (563h59m12.432633687s) [noise=true , address=145.79.227.12:4779 , piece_id=MOBPO72LIVFWAJJ7VSM6V43SYULVI7GCRECQ4MYLBZY2DRZJ5ABQ, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:51.940583931 (563h59m12.43301638s) [noise=true , address=145.79.227.23:4574 , piece_id=PKIAVX3RK56BCZNIHZLJGTX2XVBKYNLASCC4LJGFACTZCIASN6ZA, seg_pos={0 0}, upload=496159]: Uploading piece... Sep 22 02:06:52.524739183 (563h59m13.017171615s) [upload=496159]: Next returned nil segment Sep 22 02:06:52.524765374 (563h59m13.017197828s) [upload=496159]: Waiting for error group managing segments... Sep 22 02:06:52.687030567 (563h59m13.179463014s) [noise=true , address=145.79.227.16:4930 , piece_id=S5NCQZ5XUI5J4PNJXHT4YEPMBARGEQQCXUOFAX4WLMTBRU4PJR5A, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.690493974 (563h59m13.18292641s) [noise=true , address=145.79.227.40:4052 , piece_id=RPKJRMIVXJHRDPV4AECXF7SIMB7PJ7NDDR5M655UKM7EGESK44CA, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.700672844 (563h59m13.193105288s) [noise=true , address=145.79.227.13:4809 , piece_id=YCSFYNKMSP4SNBKUPGZS77UMUAVFKZKEHCZSGKDZM3MLIYDVJO4A, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.701790231 (563h59m13.194222681s) [noise=true , address=77.90.31.15:4725 , piece_id=QTTWTKGN3AD56XPDK5VTYVMEXCEPPJIOBPVMFHECMSMHJR2CWIHQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.702865557 (563h59m13.195298007s) [noise=true , address=145.79.227.39:4031 , piece_id=EQFA6RH43ZMQXUYUUTFNZLN5UPRHH7I7BKSQDYO7AQL2IMKANGJQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.706710623 (563h59m13.199143058s) [noise=true , address=145.79.227.24:4613 , piece_id=6XBZYZWLPG3NBGDDENT26MK5NJNU2R7BDZAZX5TUM4KIVD2KSQGQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.713316054 (563h59m13.205748485s) [noise=true , address=145.79.227.15:4885 , piece_id=57TWC6FA4VFH3XE7GNTOVLBQZKXCP4LSG6MZRZTDYZOBO3CLJIJA, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.714896277 (563h59m13.207328722s) [noise=true , address=145.79.227.27:4712 , piece_id=G5S5CFXX244RWRR6QUPBWC6VTNWG3QPNWZVZMAVQQF7TVGNSRTGQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.716700552 (563h59m13.209132987s) [noise=true , address=145.79.227.29:4954 , piece_id=HIQWP3ERZH2XUEY2OI3WUJVWN2CVZBA3IEWKPFT5SE6BW3QBISWA, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.719637681 (563h59m13.212070117s) [noise=true , address=145.79.227.19:4111 , piece_id=FLUQF6UKUJUW3JAFK7EE4EBJ4XNEPLITIS3OKDNPXNDMNOZ5A65Q, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.726630648 (563h59m13.219063089s) [noise=true , address=145.79.227.18:4014 , piece_id=7FJTCF52QTZHS2N43YTUPL2FXB654LZ5QM2VHDGNOMMZCE4B3KQQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:52.738224626 (563h59m13.230657063s) [noise=true , address=145.79.227.23:4574 , piece_id=PKIAVX3RK56BCZNIHZLJGTX2XVBKYNLASCC4LJGFACTZCIASN6ZA, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:06:53.199389283 (563h59m13.691821715s) [noise=true , address=64.49.13.169:4714 , piece_id=C27ZR5UAFL6N4R432UBHZPMDWJH3MILPCJRVX7SJFEZ3AZT37KRQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: <nil> Sep 22 02:07:51.368210621 (564h0m11.860643056s) [noise=true , address=145.79.227.4:4217 , piece_id=BU7KXBLIYRNOMYFC4O3DWDU3FACZDOZZRDTWNS5JUZPBPXVHOLSQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: ecclient: upload failed (node:123KLtoTMCQ9DZUjJ8JJab4D5bc7UGTt7yjVV8wDgFTozdZ5gpi, address:145.79.227.4:4217): protocol: rpc: dial tcp 145.79.227.4:4217: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:51.368251150 (564h0m11.860683583s) [noise=true , address=145.79.227.2:4131 , piece_id=SUETKESYNBCHEKOXBHLHFRIZFSMFJPLEBFTM4GRPDUO6YE432JTA, seg_pos={0 0}, upload=496159]: Done uploading piece. err: ecclient: upload failed (node:129jahwAmyPHqyYaXz7XTCK8FakBrYZw94pBKGhPzzX23yjM3RQ, address:145.79.227.2:4131): protocol: rpc: dial tcp 145.79.227.2:4131: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:51.368297293 (564h0m11.860729725s) [noise=true , address=145.79.227.10:4423 , piece_id=WQFRSEXMRVW7HNUMRJRU5TYCRIQVIB4HDNMZ4WGZOWMSHLMQEF2Q, seg_pos={0 0}, upload=496159]: Done uploading piece. err: ecclient: upload failed (node:12Rpox27kwUgBKkqhfexgNQPEomMivnFFy5bJAmNmbaFDhaaBSM, address:145.79.227.10:4423): protocol: rpc: dial tcp 145.79.227.10:4423: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:51.368315481 (564h0m11.860747913s) [noise=true , address=145.79.227.12:4779 , piece_id=MOBPO72LIVFWAJJ7VSM6V43SYULVI7GCRECQ4MYLBZY2DRZJ5ABQ, seg_pos={0 0}, upload=496159]: Done uploading piece. err: ecclient: upload failed (node:12DHKWSFHcN6FYabUR9Q8vpe7duLcyZmEPcDmqvvcQHDGm4bpvu, address:145.79.227.12:4779): protocol: rpc: dial tcp 145.79.227.12:4779: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:07:51.368417198 (564h0m11.860849632s) [noise=true , address=145.79.227.6:4261 , piece_id=SOKPD3KG2K4BCAS3LEZJ6BSFRWXW3INXXUC3D3CHU2B7IV5FQD3Q, seg_pos={0 0}, upload=496159]: Done uploading piece. err: ecclient: upload failed (node:1i1jcJdEP7Ch9uaGcjDaVxpodx55dJEHUKFbcr1GeKQ9waWjYz, address:145.79.227.6:4261): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:07:51.368443976 (564h0m11.860876425s) [seg_pos={0 0}, upload=496159]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:07:51.475111627 (564h0m11.967544083s) [upload=496159]: Done uploading segments... Sep 22 02:07:51.475150154 (564h0m11.967582596s) [upload=496159]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.10938298s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.786556695s FAIL (2025-09-20 23:25:38 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.786582387s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.772427361s FAIL (2025-09-20 23:24:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.772453339s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.812685484s FAIL (2025-09-20 23:24:03 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.812716382s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.819789569s FAIL (2025-09-20 23:23:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.819823485s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.665679883s FAIL (2025-09-20 23:22:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.665779641s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.956967477s FAIL (2025-09-20 23:22:20 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.957002135s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.766951463s FAIL (2025-09-20 23:21:19 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.766999088s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.660412316s FAIL (2025-09-20 23:20:24 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.661112497s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.901218235s FAIL (2025-09-20 23:19:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.901246247s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.753304132s FAIL (2025-09-20 23:18:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.753332006s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.766689131s FAIL (2025-09-20 23:17:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.766714774s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.737769109s FAIL (2025-09-20 23:17:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.737791697s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.771089652s FAIL (2025-09-20 23:16:53 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.771120373s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.649800918s FAIL (2025-09-20 23:15:52 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.649822104s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.761769877s FAIL (2025-09-20 23:14:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.761795125s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.918790515s FAIL (2025-09-20 23:14:25 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.920016398s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.697889935s FAIL (2025-09-20 23:13:33 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.697916567s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.731296012s FAIL (2025-09-20 23:13:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.731330869s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.887325024s FAIL (2025-09-20 23:12:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.887354123s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.070565905s FAIL (2025-09-20 23:12:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 3.070596194s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 3.109445297s FAIL (2025-09-20 23:11:00 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 3.109470997s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.767000072s FAIL (2025-09-20 23:10:17 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 139.59.144.22:51054->45.38.24.2:443: read: connection reset by peer Duration: 2.767029103s edge gateway us-select-1

1
...
372
373
374
375
376
...
3465