Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.............XX...] duration: 51.174471644s FAIL (2025-07-26 22:19:04 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 282143815s) [upload=56268]: Uploading segments... Jul 26 22:15:25.092278306 (58h39m56.282576897s) [upload=56268]: Got next segment. Inline: false Jul 26 22:15:25.092388080 (58h39m56.282686674s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:15:25.092396009 (58h39m56.282694601s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:15:25.289672213 (58h39m56.479970792s) [seg_pos={0 0}, upload=56268]: Begin upload segment... Jul 26 22:15:25.289721542 (58h39m56.480020121s) [seg_pos={0 0}, upload=56268]: Done begin upload segment. Jul 26 22:15:25.289749511 (58h39m56.480048099s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290034281 (58h39m56.480332871s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290229652 (58h39m56.480528237s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290618006 (58h39m56.480916592s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290787658 (58h39m56.481086237s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291044051 (58h39m56.481342636s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291426419 (58h39m56.481725004s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291457487 (58h39m56.481756072s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291793645 (58h39m56.482092226s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292117429 (58h39m56.482416011s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292228759 (58h39m56.482527348s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292685386 (58h39m56.482983969s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292807443 (58h39m56.483106023s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293032234 (58h39m56.483330819s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293356155 (58h39m56.483654734s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294196182 (58h39m56.48449476s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294762113 (58h39m56.485060702s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.295219747 (58h39m56.485518333s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.911445722 (58h39m57.101744301s) [upload=56268]: Next returned nil segment Jul 26 22:15:25.911516770 (58h39m57.101815354s) [upload=56268]: Waiting for error group managing segments... Jul 26 22:15:26.338859810 (58h39m57.52915838s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:26.679604432 (58h39m57.869903021s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: storage node overloaded, request limit: 2000 Jul 26 22:15:31.541563276 (58h40m2.731861854s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:33.257976637 (58h40m4.448275213s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:38.555228952 (58h40m9.745527521s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:43.258339143 (58h40m14.448637708s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:51.415097340 (58h40m22.605395913s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:54.754531467 (58h40m25.944830034s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:57.394401199 (58h40m28.58469979s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:00.401165471 (58h40m31.591464042s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:02.987276628 (58h40m34.177575197s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:08.266175908 (58h40m39.45647448s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:14.334537053 (58h40m45.524835631s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:15.443106358 (58h40m46.633404929s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860560629 (58h40m50.05085921s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860608996 (58h40m50.050907577s) [seg_pos={0 0}, upload=56268]: Segment reached optimal threshold of 14 pieces. Jul 26 22:16:18.860780330 (58h40m50.051078899s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfbCzjSwFDRrR1gPggeN8tNCLbRRWDVzCJ6mqhZsbVSe8d4s4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860813290 (58h40m50.051111863s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860926350 (58h40m50.051224918s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12h7TPnAFgxFKain3dR7H7cqRtmYg66Px2zLWgmRcoq7TpQqpoe): protocol: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection Jul 26 22:16:18.860960310 (58h40m50.051258898s) [seg_pos={0 0}, upload=56268]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:16:18.861056636 (58h40m50.051355211s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:16:18.861063534 (58h40m50.051362118s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:16:19.036721015 (58h40m50.227019605s) [upload=56268]: Done uploading segments... Jul 26 22:16:19.036747933 (58h40m50.227046513s) [upload=56268]: Upload finished. err: <nil> Duration: 51.174505302s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.468803994s FAIL (2025-07-26 22:18:31 UTC) orbiter warning: context deadline exceeded Duration: 2.468900373s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........................X.X..X..XXX....X.X.X...............] duration: 4.124191139s FAIL (2025-07-26 22:18:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 4.124233643s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 48.345917199s FAIL (2025-07-26 22:18:05 UTC) orbiter warning: context deadline exceeded Duration: 48.345946792s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.............XX..] duration: 40.57913514s FAIL (2025-07-26 22:18:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 282143815s) [upload=56268]: Uploading segments... Jul 26 22:15:25.092278306 (58h39m56.282576897s) [upload=56268]: Got next segment. Inline: false Jul 26 22:15:25.092388080 (58h39m56.282686674s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:15:25.092396009 (58h39m56.282694601s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:15:25.289672213 (58h39m56.479970792s) [seg_pos={0 0}, upload=56268]: Begin upload segment... Jul 26 22:15:25.289721542 (58h39m56.480020121s) [seg_pos={0 0}, upload=56268]: Done begin upload segment. Jul 26 22:15:25.289749511 (58h39m56.480048099s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290034281 (58h39m56.480332871s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290229652 (58h39m56.480528237s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290618006 (58h39m56.480916592s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290787658 (58h39m56.481086237s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291044051 (58h39m56.481342636s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291426419 (58h39m56.481725004s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291457487 (58h39m56.481756072s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291793645 (58h39m56.482092226s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292117429 (58h39m56.482416011s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292228759 (58h39m56.482527348s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292685386 (58h39m56.482983969s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292807443 (58h39m56.483106023s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293032234 (58h39m56.483330819s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293356155 (58h39m56.483654734s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294196182 (58h39m56.48449476s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294762113 (58h39m56.485060702s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.295219747 (58h39m56.485518333s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.911445722 (58h39m57.101744301s) [upload=56268]: Next returned nil segment Jul 26 22:15:25.911516770 (58h39m57.101815354s) [upload=56268]: Waiting for error group managing segments... Jul 26 22:15:26.338859810 (58h39m57.52915838s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:26.679604432 (58h39m57.869903021s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: storage node overloaded, request limit: 2000 Jul 26 22:15:31.541563276 (58h40m2.731861854s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:33.257976637 (58h40m4.448275213s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:38.555228952 (58h40m9.745527521s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:43.258339143 (58h40m14.448637708s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:51.415097340 (58h40m22.605395913s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:54.754531467 (58h40m25.944830034s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:57.394401199 (58h40m28.58469979s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:00.401165471 (58h40m31.591464042s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:02.987276628 (58h40m34.177575197s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:08.266175908 (58h40m39.45647448s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:14.334537053 (58h40m45.524835631s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:15.443106358 (58h40m46.633404929s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860560629 (58h40m50.05085921s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860608996 (58h40m50.050907577s) [seg_pos={0 0}, upload=56268]: Segment reached optimal threshold of 14 pieces. Jul 26 22:16:18.860780330 (58h40m50.051078899s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfbCzjSwFDRrR1gPggeN8tNCLbRRWDVzCJ6mqhZsbVSe8d4s4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860813290 (58h40m50.051111863s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860926350 (58h40m50.051224918s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12h7TPnAFgxFKain3dR7H7cqRtmYg66Px2zLWgmRcoq7TpQqpoe): protocol: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection Jul 26 22:16:18.860960310 (58h40m50.051258898s) [seg_pos={0 0}, upload=56268]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:16:18.861056636 (58h40m50.051355211s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:16:18.861063534 (58h40m50.051362118s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:16:19.036721015 (58h40m50.227019605s) [upload=56268]: Done uploading segments... Jul 26 22:16:19.036747933 (58h40m50.227046513s) [upload=56268]: Upload finished. err: <nil> Duration: 40.579176133s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........................X.X..X..XXX....X.X.X..............] duration: 2.282494824s FAIL (2025-07-26 22:17:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.282528127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................XX.X..................X.X............] duration: 3.733622289s FAIL (2025-07-26 22:17:53 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.733657551s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.795496445s FAIL (2025-07-26 22:17:40 UTC) orbiter warning: context deadline exceeded Duration: 2.795533854s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.............XX.] duration: 55.449254948s FAIL (2025-07-26 22:17:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 282143815s) [upload=56268]: Uploading segments... Jul 26 22:15:25.092278306 (58h39m56.282576897s) [upload=56268]: Got next segment. Inline: false Jul 26 22:15:25.092388080 (58h39m56.282686674s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:15:25.092396009 (58h39m56.282694601s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:15:25.289672213 (58h39m56.479970792s) [seg_pos={0 0}, upload=56268]: Begin upload segment... Jul 26 22:15:25.289721542 (58h39m56.480020121s) [seg_pos={0 0}, upload=56268]: Done begin upload segment. Jul 26 22:15:25.289749511 (58h39m56.480048099s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290034281 (58h39m56.480332871s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290229652 (58h39m56.480528237s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290618006 (58h39m56.480916592s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290787658 (58h39m56.481086237s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291044051 (58h39m56.481342636s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291426419 (58h39m56.481725004s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291457487 (58h39m56.481756072s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291793645 (58h39m56.482092226s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292117429 (58h39m56.482416011s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292228759 (58h39m56.482527348s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292685386 (58h39m56.482983969s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292807443 (58h39m56.483106023s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293032234 (58h39m56.483330819s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293356155 (58h39m56.483654734s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294196182 (58h39m56.48449476s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294762113 (58h39m56.485060702s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.295219747 (58h39m56.485518333s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.911445722 (58h39m57.101744301s) [upload=56268]: Next returned nil segment Jul 26 22:15:25.911516770 (58h39m57.101815354s) [upload=56268]: Waiting for error group managing segments... Jul 26 22:15:26.338859810 (58h39m57.52915838s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:26.679604432 (58h39m57.869903021s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: storage node overloaded, request limit: 2000 Jul 26 22:15:31.541563276 (58h40m2.731861854s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:33.257976637 (58h40m4.448275213s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:38.555228952 (58h40m9.745527521s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:43.258339143 (58h40m14.448637708s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:51.415097340 (58h40m22.605395913s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:54.754531467 (58h40m25.944830034s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:57.394401199 (58h40m28.58469979s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:00.401165471 (58h40m31.591464042s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:02.987276628 (58h40m34.177575197s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:08.266175908 (58h40m39.45647448s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:14.334537053 (58h40m45.524835631s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:15.443106358 (58h40m46.633404929s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860560629 (58h40m50.05085921s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860608996 (58h40m50.050907577s) [seg_pos={0 0}, upload=56268]: Segment reached optimal threshold of 14 pieces. Jul 26 22:16:18.860780330 (58h40m50.051078899s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfbCzjSwFDRrR1gPggeN8tNCLbRRWDVzCJ6mqhZsbVSe8d4s4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860813290 (58h40m50.051111863s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860926350 (58h40m50.051224918s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12h7TPnAFgxFKain3dR7H7cqRtmYg66Px2zLWgmRcoq7TpQqpoe): protocol: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection Jul 26 22:16:18.860960310 (58h40m50.051258898s) [seg_pos={0 0}, upload=56268]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:16:18.861056636 (58h40m50.051355211s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:16:18.861063534 (58h40m50.051362118s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:16:19.036721015 (58h40m50.227019605s) [upload=56268]: Done uploading segments... Jul 26 22:16:19.036747933 (58h40m50.227046513s) [upload=56268]: Upload finished. err: <nil> Duration: 55.449279804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...........................X.X..X..XXX....X.X.X.............] duration: 7.88564884s FAIL (2025-07-26 22:16:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 7.885696341s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................XX.X..................X.X...........] duration: 2.400149613s FAIL (2025-07-26 22:16:48 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.40019299s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 25.949007458s FAIL (2025-07-26 22:16:44 UTC) orbiter warning: context deadline exceeded Duration: 25.949041157s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.473664475s FAIL (2025-07-26 22:16:27 UTC) orbiter warning: context deadline exceeded Duration: 2.47370953s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.............XX] duration: 1m0.003574737s FAIL (2025-07-26 22:16:24 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 282143815s) [upload=56268]: Uploading segments... Jul 26 22:15:25.092278306 (58h39m56.282576897s) [upload=56268]: Got next segment. Inline: false Jul 26 22:15:25.092388080 (58h39m56.282686674s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:15:25.092396009 (58h39m56.282694601s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:15:25.289672213 (58h39m56.479970792s) [seg_pos={0 0}, upload=56268]: Begin upload segment... Jul 26 22:15:25.289721542 (58h39m56.480020121s) [seg_pos={0 0}, upload=56268]: Done begin upload segment. Jul 26 22:15:25.289749511 (58h39m56.480048099s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290034281 (58h39m56.480332871s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290229652 (58h39m56.480528237s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290618006 (58h39m56.480916592s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.290787658 (58h39m56.481086237s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291044051 (58h39m56.481342636s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291426419 (58h39m56.481725004s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291457487 (58h39m56.481756072s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.291793645 (58h39m56.482092226s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292117429 (58h39m56.482416011s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292228759 (58h39m56.482527348s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292685386 (58h39m56.482983969s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.292807443 (58h39m56.483106023s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293032234 (58h39m56.483330819s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.293356155 (58h39m56.483654734s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294196182 (58h39m56.48449476s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.294762113 (58h39m56.485060702s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.295219747 (58h39m56.485518333s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Uploading piece... Jul 26 22:15:25.911445722 (58h39m57.101744301s) [upload=56268]: Next returned nil segment Jul 26 22:15:25.911516770 (58h39m57.101815354s) [upload=56268]: Waiting for error group managing segments... Jul 26 22:15:26.338859810 (58h39m57.52915838s) [noise=true , address=79.127.219.38:4015 , piece_id=7HGP67WRTBUSKBGKXZUUM4KTYV5HZ7PWFJTWDYT6JDHNHMNK5YZA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:26.679604432 (58h39m57.869903021s) [noise=true , address=64.49.13.154:4935 , piece_id=V3LIMNZUS4JQ4INWB3OHPJMW65HSRRX6EC6ZWP5HMQYX2ZM6PTFQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: storage node overloaded, request limit: 2000 Jul 26 22:15:31.541563276 (58h40m2.731861854s) [noise=true , address=79.127.201.215:4354 , piece_id=QVVRNI2U2GRG2O4HZR72SFDL7PKTSQCB7VRS2ONJOSVWZOKH5VFA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:33.257976637 (58h40m4.448275213s) [noise=true , address=156.146.43.225:4448 , piece_id=Z2V37XOL33XL5JXPXEEKGR3T6WYQZ6AUVLB7LZOASHOLLSRBEI3A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:38.555228952 (58h40m9.745527521s) [noise=true , address=104.36.82.9:4013 , piece_id=NLHLY7DXO2AP2NPZPE6XYCBRCYWAVBOSCFOIKB74TU36U5RSZDRQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:43.258339143 (58h40m14.448637708s) [noise=true , address=104.36.82.53:4128 , piece_id=JAFMMVVJCYEVRLBTWORRPY5Q4GICBJEUXQVUMJX4SY5VS74IKBWQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:51.415097340 (58h40m22.605395913s) [noise=true , address=104.36.82.49:4978 , piece_id=FRGSTLW2KFJVCS45UCNHGS6XBCTKVQSNIUF4CDZM2ZJTNHWH4FEA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:54.754531467 (58h40m25.944830034s) [noise=true , address=104.36.82.46:4481 , piece_id=WZPLZZ7N6MCRMAAMUG4X6ROKOSGSIBVTUEJZEG5TC7CYBWMYTGEQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:15:57.394401199 (58h40m28.58469979s) [noise=true , address=104.36.82.41:4785 , piece_id=MP2PEOXYSGB323EKB37QATIBBCSNV4TY2H6AC7C7SQ5OTCNBCIMA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:00.401165471 (58h40m31.591464042s) [noise=true , address=104.36.82.52:4100 , piece_id=M4OQO3QK5ADPGJYR7F4QDVWDZDM4GJLQY7WHNPD3VK43VVPJO6ZQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:02.987276628 (58h40m34.177575197s) [noise=true , address=104.36.82.51:4072 , piece_id=PRP6GWSBIB5AVT74W6VMVO5WOBNOQC5GNZE5Q7676NGG4JAP4QBQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:08.266175908 (58h40m39.45647448s) [noise=true , address=104.36.82.35:4592 , piece_id=J6V6CJLF5SJISNRPUYTKCOTNSFATPG3LDPLDVVU433PPD2CUFBDA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:14.334537053 (58h40m45.524835631s) [noise=true , address=104.36.82.17:4351 , piece_id=7M7A6MTNZJM7YH2RXJHWPRGYPFC5HMMYXPZ2SM5T7F2SUCFWW7FQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:15.443106358 (58h40m46.633404929s) [noise=true , address=104.36.82.55:4225 , piece_id=BZWX762BYK74ZBAO3AEGKYU6TAUS32VBI2COE5RDSVLQKGFI2QKA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860560629 (58h40m50.05085921s) [noise=true , address=104.36.82.5:4847 , piece_id=5BVD264I2NQODAWWCZKZK7A6BEQLGFXTMMWWFDCF4ZHQQ2HHMRMQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: <nil> Jul 26 22:16:18.860608996 (58h40m50.050907577s) [seg_pos={0 0}, upload=56268]: Segment reached optimal threshold of 14 pieces. Jul 26 22:16:18.860780330 (58h40m50.051078899s) [noise=true , address=104.36.82.47:4524 , piece_id=WHTP76SZMGO7UQJLV6NVV3PRICSGP6TLF26IBBFMZQMMC6XGCUCQ, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xfbCzjSwFDRrR1gPggeN8tNCLbRRWDVzCJ6mqhZsbVSe8d4s4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860813290 (58h40m50.051111863s) [noise=true , address=104.36.82.8:4996 , piece_id=CVINCYWKSI54ETITMKOA6BUD7M4ZIL4AI5SNCBFPERW5VXYEZXHA, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r6XcE2fb6YxU6kez4UMPeq82jNPi7YiRNmi33tQcTVYMYrMV4): protocol: expected piece hash; context canceled; EOF Jul 26 22:16:18.860926350 (58h40m50.051224918s) [noise=true , address=104.36.82.21:4481 , piece_id=5OENCJXFGCZ5NSEWL3IWYY62Z4GKJZD3PJYBBMES6KYE4DYAOC4A, seg_pos={0 0}, upload=56268]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12h7TPnAFgxFKain3dR7H7cqRtmYg66Px2zLWgmRcoq7TpQqpoe): protocol: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60532->104.36.82.21:4481: use of closed network connection Jul 26 22:16:18.860960310 (58h40m50.051258898s) [seg_pos={0 0}, upload=56268]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:16:18.861056636 (58h40m50.051355211s) [upload=56268]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:16:18.861063534 (58h40m50.051362118s) [upload=56268]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:16:19.036721015 (58h40m50.227019605s) [upload=56268]: Done uploading segments... Jul 26 22:16:19.036747933 (58h40m50.227046513s) [upload=56268]: Upload finished. err: <nil> Duration: 1m0.003598504s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 43.06363654s FAIL (2025-07-26 22:16:08 UTC) orbiter warning: context deadline exceeded Duration: 43.063677614s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [............................X.X..X..XXX....X.X.X............] duration: 2.347752688s FAIL (2025-07-26 22:15:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.347780567s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.632570177s FAIL (2025-07-26 22:15:48 UTC) orbiter warning: context deadline exceeded Duration: 2.632591115s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................XX.X..................X.X..........] duration: 2.464260314s FAIL (2025-07-26 22:15:34 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.464291177s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001509766s FAIL (2025-07-26 22:15:25 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001536789s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.............X] duration: 1m0.00378925s FAIL (2025-07-26 22:15:24 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 6.12705931s) [upload=56250]: Done starting upload Jul 26 22:14:24.936904844 (58h38m56.127203409s) [upload=56250]: Uploading segments... Jul 26 22:14:24.937250373 (58h38m56.127548947s) [upload=56250]: Got next segment. Inline: false Jul 26 22:14:24.937312239 (58h38m56.12761082s) [upload=56250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:14:24.937320937 (58h38m56.127619524s) [upload=56250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:14:25.149106609 (58h38m56.339405177s) [seg_pos={0 0}, upload=56250]: Begin upload segment... Jul 26 22:14:25.149167901 (58h38m56.33946648s) [seg_pos={0 0}, upload=56250]: Done begin upload segment. Jul 26 22:14:25.149194638 (58h38m56.339493222s) [noise=true , address=104.36.82.12:4162 , piece_id=CRBOQUO6JNO4LZFPXS7TF7PEB3PM5A4PD4LDFGURV25OKYQDNTQA, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.149708121 (58h38m56.340006703s) [noise=true , address=79.127.205.246:4275 , piece_id=KCHISVNP43SWQBSJ3MRE4ZOAEJQLPOS6QSH6PDWUHBR7GRKEPFGQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.150022555 (58h38m56.340321148s) [noise=true , address=104.36.82.42:4816 , piece_id=QXFCMKZZ26T26BD5VEDPQXAEHARZHGAHUEI4VXI36GXGRFA6UUPQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.150220739 (58h38m56.340519322s) [noise=true , address=64.49.13.137:4500 , piece_id=3ABWBAFKJSHJOCR4GT5G2EBPXFPLQIM4CMUNZF2AGFLVDKDZYO4Q, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.150468821 (58h38m56.340767406s) [noise=true , address=79.127.205.234:4613 , piece_id=GMGPRXCHH5YPBGGMGHUTQ6FSCITR573XRKBRKBGRM6CQDBTBZAJQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.150676528 (58h38m56.340975121s) [noise=true , address=104.36.82.52:4091 , piece_id=RZ3X7FFBSZDPRGU2GAHL5IQCFP3M7RZHPS4Z5Y4RX2S7IGTKU5AA, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.150832620 (58h38m56.34113121s) [noise=true , address=104.36.82.13:4206 , piece_id=6XJ4MJUJDGOEMKYOLJRBIQZ7C3FITSOIW6YV24NK7QB5LGHYMKWA, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.151197616 (58h38m56.341496206s) [noise=true , address=104.36.82.47:4527 , piece_id=QLQ4ESJN6MVDPUM2HDUGSEGRJVUVMWFOM7MGZNEPZUBDDPQMYBNQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.151236791 (58h38m56.341535378s) [noise=true , address=104.36.82.6:4902 , piece_id=TXTMOQOG4XY427MGIU5NPA464OJXZE4OZZI3FVFK4YBHMLVQ234Q, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.151690946 (58h38m56.341989517s) [noise=true , address=104.36.82.51:4078 , piece_id=VJYQM2T6WKCWGGFAMLNWTM75AQGUIAJVBZFP2X36LTFFWN5O3QCA, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.151776544 (58h38m56.342075136s) [noise=true , address=64.49.13.138:4482 , piece_id=R3C6FZCNU4UKQMFJDHUOSVN4EX6YUVLVH2YHON2J5SWN7262QXOQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.152285938 (58h38m56.342584528s) [noise=true , address=104.36.82.9:4028 , piece_id=ZZ3YBKTVH3LZ7TFB7JZHQ6MBR5OWM36ZPRFTYDQHETPMG7U7PMAQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.152656991 (58h38m56.34295558s) [noise=true , address=104.36.82.56:4245 , piece_id=MAJFBCSXB5ZVDA3DCABQRHG2XZQS5UG725F6YGPGJHRBQNPQAMIQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.153022511 (58h38m56.3433211s) [noise=true , address=104.36.82.35:4600 , piece_id=ZOIM3PMYSW63GCJIQMLZPQGJVUYVH7WAVC7EKRGFXO4JHFXTKFOQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.153984585 (58h38m56.344283175s) [noise=true , address=104.36.82.55:4219 , piece_id=MWIFSVMDTL7ZCYKT46EILOTEFRZTIX5ZW6QTENRTB5AXLPHS3CTQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.154462250 (58h38m56.344760837s) [noise=true , address=104.36.82.13:4197 , piece_id=SGWLOJ4FDG7BM5OBEGDKX6TVQQQ3WWH7BKX436OJMYBACGF4JIRA, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.155020407 (58h38m56.345318998s) [noise=true , address=77.90.31.17:4786 , piece_id=UYD6FZW4RAZ3QENZRBFLNULXBLVXT6OEDCSX4PLHUBUPG4GW2HOQ, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.156492391 (58h38m56.346790973s) [noise=true , address=104.36.82.46:4482 , piece_id=FQX62PCMT7EI2N3WHSE5QJNCQI65UFWUGHWY7XOO5M7M5BN3QJ3A, seg_pos={0 0}, upload=56250]: Uploading piece... Jul 26 22:14:25.924191018 (58h38m57.114489607s) [upload=56250]: Next returned nil segment Jul 26 22:14:25.924217631 (58h38m57.11451622s) [upload=56250]: Waiting for error group managing segments... Jul 26 22:14:26.346134316 (58h38m57.536432918s) [noise=true , address=79.127.205.246:4275 , piece_id=KCHISVNP43SWQBSJ3MRE4ZOAEJQLPOS6QSH6PDWUHBR7GRKEPFGQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:26.439330981 (58h38m57.629629551s) [noise=true , address=79.127.205.234:4613 , piece_id=GMGPRXCHH5YPBGGMGHUTQ6FSCITR573XRKBRKBGRM6CQDBTBZAJQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:28.897762167 (58h39m0.088060753s) [noise=true , address=64.49.13.137:4500 , piece_id=3ABWBAFKJSHJOCR4GT5G2EBPXFPLQIM4CMUNZF2AGFLVDKDZYO4Q, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:31.447799570 (58h39m2.638098149s) [noise=true , address=77.90.31.17:4786 , piece_id=UYD6FZW4RAZ3QENZRBFLNULXBLVXT6OEDCSX4PLHUBUPG4GW2HOQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:33.532763064 (58h39m4.723061642s) [noise=true , address=104.36.82.35:4600 , piece_id=ZOIM3PMYSW63GCJIQMLZPQGJVUYVH7WAVC7EKRGFXO4JHFXTKFOQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:33.613465873 (58h39m4.803764465s) [noise=true , address=64.49.13.138:4482 , piece_id=R3C6FZCNU4UKQMFJDHUOSVN4EX6YUVLVH2YHON2J5SWN7262QXOQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:46.151033485 (58h39m17.341332071s) [noise=true , address=104.36.82.6:4902 , piece_id=TXTMOQOG4XY427MGIU5NPA464OJXZE4OZZI3FVFK4YBHMLVQ234Q, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:58.656642048 (58h39m29.846940627s) [noise=true , address=104.36.82.13:4197 , piece_id=SGWLOJ4FDG7BM5OBEGDKX6TVQQQ3WWH7BKX436OJMYBACGF4JIRA, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:14:59.935516069 (58h39m31.12581465s) [noise=true , address=104.36.82.42:4816 , piece_id=QXFCMKZZ26T26BD5VEDPQXAEHARZHGAHUEI4VXI36GXGRFA6UUPQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:02.007665395 (58h39m33.197963996s) [noise=true , address=104.36.82.13:4206 , piece_id=6XJ4MJUJDGOEMKYOLJRBIQZ7C3FITSOIW6YV24NK7QB5LGHYMKWA, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:05.399384897 (58h39m36.589683476s) [noise=true , address=104.36.82.55:4219 , piece_id=MWIFSVMDTL7ZCYKT46EILOTEFRZTIX5ZW6QTENRTB5AXLPHS3CTQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:06.767971548 (58h39m37.958270113s) [noise=true , address=104.36.82.52:4091 , piece_id=RZ3X7FFBSZDPRGU2GAHL5IQCFP3M7RZHPS4Z5Y4RX2S7IGTKU5AA, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:08.753721586 (58h39m39.944020157s) [noise=true , address=104.36.82.56:4245 , piece_id=MAJFBCSXB5ZVDA3DCABQRHG2XZQS5UG725F6YGPGJHRBQNPQAMIQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:16.987519657 (58h39m48.177818227s) [noise=true , address=104.36.82.46:4482 , piece_id=FQX62PCMT7EI2N3WHSE5QJNCQI65UFWUGHWY7XOO5M7M5BN3QJ3A, seg_pos={0 0}, upload=56250]: Done uploading piece. err: <nil> Jul 26 22:15:16.987544526 (58h39m48.177843097s) [seg_pos={0 0}, upload=56250]: Segment reached optimal threshold of 14 pieces. Jul 26 22:15:16.987673974 (58h39m48.177972553s) [noise=true , address=104.36.82.9:4028 , piece_id=ZZ3YBKTVH3LZ7TFB7JZHQ6MBR5OWM36ZPRFTYDQHETPMG7U7PMAQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XJfiw7ni2wahnKMgWbVGXF14vU9yzVjEquAA8nJus251cHECV): protocol: expected piece hash; context canceled; EOF Jul 26 22:15:16.987779615 (58h39m48.178078195s) [noise=true , address=104.36.82.47:4527 , piece_id=QLQ4ESJN6MVDPUM2HDUGSEGRJVUVMWFOM7MGZNEPZUBDDPQMYBNQ, seg_pos={0 0}, upload=56250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17xmkbFoj3PTJnAygsjRAcfdd17AfSkPP2xdZp68GGPrvfMuU6): protocol: expected piece hash; context canceled; EOF Jul 26 22:15:16.987958517 (58h39m48.178257097s) [noise=true , address=104.36.82.12:4162 , piece_id=CRBOQUO6JNO4LZFPXS7TF7PEB3PM5A4PD4LDFGURV25OKYQDNTQA, seg_pos={0 0}, upload=56250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1yxXQkfWVGwBuptpPjFgjhfTraAmE3WqtS3WEDbhQsJftjeaxV): protocol: expected piece hash; context canceled; EOF Jul 26 22:15:16.988059209 (58h39m48.178357789s) [noise=true , address=104.36.82.51:4078 , piece_id=VJYQM2T6WKCWGGFAMLNWTM75AQGUIAJVBZFP2X36LTFFWN5O3QCA, seg_pos={0 0}, upload=56250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tzgtenV6us5ssD7xWGpZmVUoWEfbFhBNewbK9irHtZPkyDtRd): protocol: expected piece hash; write tcp 139.59.144.22:34478->104.36.82.51:4078: use of closed network connection; write tcp 139.59.144.22:34478->104.36.82.51:4078: use of closed network connection; EOF Jul 26 22:15:16.988076480 (58h39m48.178375049s) [seg_pos={0 0}, upload=56250]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:15:16.988472695 (58h39m48.178771284s) [upload=56250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:15:16.988482643 (58h39m48.178781208s) [upload=56250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:15:17.144508887 (58h39m48.334807477s) [upload=56250]: Done uploading segments... Jul 26 22:15:17.144550602 (58h39m48.33484919s) [upload=56250]: Upload finished. err: <nil> Duration: 1m0.003808265s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................X.X..X..XXX....X.X.X...........] duration: 2.426755998s FAIL (2025-07-26 22:14:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.42679655s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.591775033s FAIL (2025-07-26 22:14:47 UTC) orbiter warning: context deadline exceeded Duration: 2.59179486s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................X.X..X..XXX....X.X.X..........] duration: 2.347165862s FAIL (2025-07-26 22:14:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.347197288s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 40.983041743s FAIL (2025-07-26 22:14:24 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 40.983066127s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................XX.X..................X.X.........] duration: 2.255094274s FAIL (2025-07-26 22:14:22 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.25512847s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................X.X..X..XXX....X.X.X.........] duration: 11.910547982s FAIL (2025-07-26 22:14:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 11.910580482s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.694231808s FAIL (2025-07-26 22:14:03 UTC) orbiter warning: context deadline exceeded Duration: 2.694254187s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 47.268051431s FAIL (2025-07-26 22:13:43 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 47.268081583s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................X.X..X..XXX....X.X.X........] duration: 2.344748484s FAIL (2025-07-26 22:13:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.344784683s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.637021158s FAIL (2025-07-26 22:13:21 UTC) orbiter warning: context deadline exceeded Duration: 2.637052021s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................XX.X..................X.X........] duration: 2.312330168s FAIL (2025-07-26 22:13:14 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.312360228s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................X.X..X..XXX....X.X.X.......] duration: 2.332401075s FAIL (2025-07-26 22:13:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.332433534s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.522139056s FAIL (2025-07-26 22:12:57 UTC) orbiter warning: context deadline exceeded Duration: 2.5221669s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.775352783s FAIL (2025-07-26 22:12:42 UTC) orbiter warning: context deadline exceeded Duration: 2.775369219s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 29.477817697s FAIL (2025-07-26 22:12:33 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 29.477841914s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................XX.X..................X.X.......] duration: 2.353969486s FAIL (2025-07-26 22:12:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.354002041s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 36.052532283s FAIL (2025-07-26 22:11:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 36.052559668s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................................X.X..X..XXX....X.X.X......] duration: 3.461342399s FAIL (2025-07-26 22:11:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 3.46137141s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 3.090385051s FAIL (2025-07-26 22:11:50 UTC) orbiter warning: context deadline exceeded Duration: 3.090411234s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................X.X..X..XXX....X.X.X.....] duration: 2.31735562s FAIL (2025-07-26 22:11:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.317424565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 39.349158161s FAIL (2025-07-26 22:10:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 39.349182295s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................XX.X..................X.X......] duration: 2.319496412s FAIL (2025-07-26 22:10:52 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.319527043s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.792960475s FAIL (2025-07-26 22:10:40 UTC) orbiter warning: context deadline exceeded Duration: 2.793010225s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....................................X.X..X..XXX....X.X.X....] duration: 2.342280132s FAIL (2025-07-26 22:10:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.342306475s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 40.18744307s FAIL (2025-07-26 22:10:19 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 40.187474684s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................XX.X..................X.X.....] duration: 25.602523396s FAIL (2025-07-26 22:10:06 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 25.60255175s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.256800269s FAIL (2025-07-26 22:09:55 UTC) orbiter warning: context deadline exceeded Duration: 3.256827928s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 51.364001017s FAIL (2025-07-26 22:09:39 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 51.364044083s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.662345468s FAIL (2025-07-26 22:09:28 UTC) orbiter warning: context deadline exceeded Duration: 2.662382927s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....................................X.X..X..XXX....X.X.X...] duration: 2.257721801s FAIL (2025-07-26 22:09:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 2.25775002s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)