Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.429070744s FAIL (2025-10-23 06:06:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.429112897s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.525139934s FAIL (2025-10-23 06:05:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.525178942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.329261703s FAIL (2025-10-23 06:04:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.32931031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.457566536s FAIL (2025-10-23 06:04:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.457596894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.538711125s FAIL (2025-10-23 06:02:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.538748576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.396248752s FAIL (2025-10-23 06:01:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.396298256s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.384451678s FAIL (2025-10-23 06:00:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.384493014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.405336308s FAIL (2025-10-23 05:59:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.405370056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.444649651s FAIL (2025-10-23 05:59:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.444684407s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.406518587s FAIL (2025-10-23 05:58:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.406611299s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.397157392s FAIL (2025-10-23 05:57:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.397191471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.374355327s FAIL (2025-10-23 05:57:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.37438928s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.42447657s FAIL (2025-10-23 05:56:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.424509589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.385491083s FAIL (2025-10-23 05:55:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.385528089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.376630007s FAIL (2025-10-23 05:54:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.376669911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.422959411s FAIL (2025-10-23 05:54:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.422997558s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.33157948s FAIL (2025-10-23 05:53:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.33162182s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.336745384s FAIL (2025-10-23 05:52:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.336774923s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.336201758s FAIL (2025-10-23 05:51:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.336233694s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.429481721s FAIL (2025-10-23 05:50:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.429518549s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.411079529s FAIL (2025-10-23 05:49:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.411116921s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.388131698s FAIL (2025-10-23 05:49:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.388167507s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.414190172s FAIL (2025-10-23 05:48:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.414222528s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.378788551s FAIL (2025-10-23 05:47:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.378828214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.395800261s FAIL (2025-10-23 05:46:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.395833918s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.404672394s FAIL (2025-10-23 05:45:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.404699825s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.362087459s FAIL (2025-10-23 05:45:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.362115569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.463432634s FAIL (2025-10-23 05:44:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.463486537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.344259675s FAIL (2025-10-23 05:43:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.344286888s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.429174657s FAIL (2025-10-23 05:42:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.429202632s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.323659942s FAIL (2025-10-23 05:41:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.323693219s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.303843679s FAIL (2025-10-23 05:41:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.303879122s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.378163659s FAIL (2025-10-23 05:40:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.37819582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.401271268s FAIL (2025-10-23 05:39:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.401303815s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.61206844s FAIL (2025-10-23 05:38:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.612101993s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.310721795s FAIL (2025-10-23 05:36:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.310755269s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.43497313s FAIL (2025-10-23 05:35:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.435013148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.388332118s FAIL (2025-10-23 05:34:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.38836639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.416613548s FAIL (2025-10-23 05:34:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.416651063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.35033194s FAIL (2025-10-23 05:33:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.350399146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.404545534s FAIL (2025-10-23 05:32:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.404584731s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.338890935s FAIL (2025-10-23 05:32:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.338951171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.355015146s FAIL (2025-10-23 05:31:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.355050423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.374298569s FAIL (2025-10-23 05:31:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.374335027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.441881619s FAIL (2025-10-23 05:30:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.441904769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m33.520062089s FAIL (2025-10-23 05:30:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 23 05:28:21.271840687 (711h58m9.075305279s) [upload=626420]: Starting upload Oct 23 05:28:21.271922292 (711h58m9.075386893s) [upload=626420]: Done starting upload Oct 23 05:28:21.272260425 (711h58m9.075725018s) [upload=626420]: Uploading segments... Oct 23 05:28:21.272314976 (711h58m9.075779634s) [upload=626420]: Got next segment. Inline: false Oct 23 05:28:21.272348622 (711h58m9.075813227s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 05:28:21.272357611 (711h58m9.075822212s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 05:28:21.385745184 (711h58m9.189209787s) [seg_pos={0 0}, upload=626420]: Begin upload segment... Oct 23 05:28:21.385814613 (711h58m9.189279212s) [seg_pos={0 0}, upload=626420]: Done begin upload segment. Oct 23 05:28:21.385852222 (711h58m9.189316822s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.386760323 (711h58m9.190224924s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387265926 (711h58m9.190730528s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.387711840 (711h58m9.191176441s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388130445 (711h58m9.191595046s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.388737138 (711h58m9.192201749s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.389638813 (711h58m9.193103415s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390122129 (711h58m9.193586733s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390531767 (711h58m9.193996361s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.390930529 (711h58m9.194395131s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391297631 (711h58m9.194762225s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391610238 (711h58m9.195074832s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.391949430 (711h58m9.195414036s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392331393 (711h58m9.195795993s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.392782012 (711h58m9.196246618s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:21.393190875 (711h58m9.196655481s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Uploading piece... Oct 23 05:28:22.007710883 (711h58m9.811175482s) [upload=626420]: Next returned nil segment Oct 23 05:28:22.007735911 (711h58m9.811200513s) [upload=626420]: Waiting for error group managing segments... Oct 23 05:28:22.132881572 (711h58m9.936346164s) [noise=true , address=145.79.227.17:4972 , piece_id=C25GBFOOVJQDDFCJY356O2BDPDEZWN3WTQXFVABKA6XCGSLKLAWA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.136737569 (711h58m9.940202161s) [noise=true , address=145.79.227.19:4079 , piece_id=5MVA2XYHTRWMKOMG2X63TS2PKQDAHES7H3AZV3KZTOLVJHO3LZKQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.137875682 (711h58m9.941340262s) [noise=true , address=145.79.227.5:4223 , piece_id=JBD74APS2AGC2TIQ5R4DRJ4VZK4WK4Q4LNCQBRXMI2KSYQ44SPAA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.138885882 (711h58m9.942350467s) [noise=true , address=145.79.227.10:4403 , piece_id=VNR27NTN6A6KDA3MEC6R7JBU75YOAMEGBKPBPEVZH45EAIBCJY7Q, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.140507461 (711h58m9.943972055s) [noise=true , address=145.79.227.22:4509 , piece_id=QTFNLMANSO2JWQWE6IOJSZAEEBGHVJJNUYA5W5PIPL2RQIQ7JSOA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.141672043 (711h58m9.945136629s) [noise=true , address=145.79.227.3:4149 , piece_id=AQZ5ZUD3BZ7Q2W6D4JIY2CAEOBTSZY5KI6YSNFAROACXBQOW2RCA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142120535 (711h58m9.945585131s) [noise=true , address=77.90.31.14:4663 , piece_id=655EIFLSZ3AGSZFEFCYKE57PN4YV5M6VV3QTNBZRA3AZLVZCBQYA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.142533104 (711h58m9.945997694s) [noise=true , address=145.79.227.24:4582 , piece_id=7KR6ZCKTS2QTAWK2BRZJAKLWN7NGHK5ZKZRKQGWKXIXJRXO3TWJA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.143461348 (711h58m9.94692594s) [noise=true , address=145.79.227.21:4474 , piece_id=OYQ6QT6FRP6AYM4OTVCFV654I76W7AHESPOUAERKXCH4CPDG5DMA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.144839855 (711h58m9.948304439s) [noise=true , address=145.79.227.13:4801 , piece_id=TGDZ6HKAQM7R5SIAQECSSMLBMOZGHFXBUOMMYCFLV7UCWBPYUB4A, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.148960207 (711h58m9.952424788s) [noise=true , address=145.79.227.2:4113 , piece_id=GPPPQNVH4CKCLLVM56RAIVNP3GD4WLPGWXYIHUIEUMCQZZE7I3YQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.158392133 (711h58m9.961856718s) [noise=true , address=145.79.227.8:4331 , piece_id=CKLRMNWQE2B7ATAQHPT5XW5GCPLNLIVTQN6WS4LR673HWFAUNOAQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161292941 (711h58m9.964757539s) [noise=true , address=145.79.227.11:4729 , piece_id=XQRAINJTD2IWR3JBLYKB5MRNI757RARRHNKZEIM4ZJZVM2RPVXEQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: <nil> Oct 23 05:28:22.161320542 (711h58m9.964785145s) [seg_pos={0 0}, upload=626420]: Segment reached optimal threshold of 13 pieces. Oct 23 05:28:22.161449162 (711h58m9.964913753s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161462946 (711h58m9.964927547s) [noise=true , address=77.90.31.21:4877 , piece_id=3IXHI2NXTCHSCZNGUIYOPZZEDXOO543YSBQGLFAB2UJF747HE3DQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161577412 (711h58m9.965042005s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161589499 (711h58m9.965054097s) [noise=true , address=145.79.227.8:4329 , piece_id=DX25HUPP7FCRBOY72XKXDWBNPOVLLAMW3MEEKI67RV4J3DFMBOGQ, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161660630 (711h58m9.965125219s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 05:28:22.161669452 (711h58m9.965134047s) [noise=true , address=145.79.227.23:4545 , piece_id=ULQICQ2DTHIQHFOC4DMTAAVKAXF2IKOA2GT2YLZPLLF2K7JSR4TA, seg_pos={0 0}, upload=626420]: Long tail cancellation due to optimal threshold Oct 23 05:28:22.161687904 (711h58m9.965152499s) [seg_pos={0 0}, upload=626420]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 05:28:22.161771876 (711h58m9.965236471s) [upload=626420]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 05:28:22.161778797 (711h58m9.965243396s) [upload=626420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 05:30:54.501745374 (712h0m42.305209968s) [upload=626420]: Done uploading segments... Oct 23 05:30:54.501775162 (712h0m42.305239756s) [upload=626420]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m33.520089428s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 85.969226ms remote: "tcp:15.160.106.203:443" response headers: Content-Type=text/html Content-Length=175 Vary=Accept-Encoding Set-Cookie=_cfuvid=QjElLuwlnze.pF4dqOmvz6UtrbJ7te1yBAA9PXDmXJo-1761184309690-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Cf-Ray=992d976f89fa4f74-MXP Alt-Svc=h3=":443"; ma=86400 X-Cluster-Name=eu-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 01:51:49 GMT target: https://www.storj.io/ FAIL (2025-10-23 02:03:26 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body>\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" Duration: 86.007207ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 120.636287ms remote: "tcp:15.160.106.203:443" response headers: Content-Type=text/html Content-Length=175 Vary=Accept-Encoding Set-Cookie=_cfuvid=QjElLuwlnze.pF4dqOmvz6UtrbJ7te1yBAA9PXDmXJo-1761184309690-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Cf-Ray=992d976f89fa4f74-MXP Alt-Svc=h3=":443"; ma=86400 X-Cluster-Name=eu-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 01:51:49 GMT target: https://www.storj.io/ FAIL (2025-10-23 02:02:29 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body>\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" Duration: 120.668223ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 107.551648ms remote: "tcp:15.160.106.203:443" response headers: Content-Type=text/html Content-Length=175 Vary=Accept-Encoding Set-Cookie=_cfuvid=QjElLuwlnze.pF4dqOmvz6UtrbJ7te1yBAA9PXDmXJo-1761184309690-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Cf-Ray=992d976f89fa4f74-MXP Alt-Svc=h3=":443"; ma=86400 X-Cluster-Name=eu-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 01:51:49 GMT target: https://www.storj.io/ FAIL (2025-10-23 02:01:23 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body>\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" Duration: 107.640209ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 85.896466ms remote: "tcp:15.160.106.203:443" response headers: Content-Type=text/html Content-Length=175 Vary=Accept-Encoding Set-Cookie=_cfuvid=QjElLuwlnze.pF4dqOmvz6UtrbJ7te1yBAA9PXDmXJo-1761184309690-0.0.1.1-604800000; path=/; domain=.cdn.webflow.com; HttpOnly; Secure; SameSite=None Cf-Ray=992d976f89fa4f74-MXP Alt-Svc=h3=":443"; ma=86400 X-Cluster-Name=eu-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 01:51:49 GMT target: https://www.storj.io/ FAIL (2025-10-23 02:00:37 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body>\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>cloudflare</center>\r\n</body>\r\n</html>\r\n" Duration: 85.935288ms core website

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)