Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 2.345084588s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:13:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.345117801s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....X...X..................................................] duration: 1.603835944s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:13:49 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.603858127s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 8 out of 60 is failed (threshold=3) [..X.X.X.XX..XX.X............................................] duration: 295.790697ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:13:48 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 295.822303ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X....X........X...........................................] duration: 1.110269459s trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:13:42 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.110306225s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 2.305246981s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:13:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.305274908s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 931.625315ms trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:13:37 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 931.665395ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.410041456s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:13:34 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.410070811s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 8 out of 60 is failed (threshold=3) [...X.X.X.XX..XX.X...........................................] duration: 286.657137ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:13:26 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 286.671664ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X...X.................................................] duration: 1.554659761s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:13:12 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.554686588s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X...X................................................] duration: 1.601082986s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:12:53 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.601198653s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 2.540191566s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:12:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.540216506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 893.905711ms trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:12:43 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 893.931207ms core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 8 out of 60 is failed (threshold=3) [....X.X.X.XX..XX.X..........................................] duration: 298.844977ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:12:42 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 298.895789ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X....X........X..........................................] duration: 1.038747201s trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:12:40 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.038773283s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.638780953s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:12:29 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.638810711s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 8 out of 60 is failed (threshold=3) [.....X.X.X.XX..XX.X.........................................] duration: 272.489818ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:12:18 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 272.515839ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X....X........X.........................................] duration: 1.045578275s trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:12:09 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.045606307s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 2.307179812s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:12:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.307217484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 906.068101ms trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:12:06 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 906.094125ms core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 8 out of 60 is failed (threshold=3) [......X.X.X.XX..XX.X........................................] duration: 270.33437ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:11:57 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 270.368793ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X...X...............................................] duration: 1.683919478s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:11:57 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.683945588s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.437174653s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:11:47 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.437199145s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X....X........X........................................] duration: 1.011858958s trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:11:41 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.011950389s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........X...X..............................................] duration: 1.709036215s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:11:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.709065905s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 2.524006839s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:11:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.524042648s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 9 out of 60 is failed (threshold=3) [X......X.X.X.XX..XX.X.......................................] duration: 295.991591ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:11:22 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 296.014644ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.53308727s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:11:18 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.533127928s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...........X...............................................] duration: 1.041745257s trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:11:04 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.041763417s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 9 out of 60 is failed (threshold=3) [.X......X.X.X.XX..XX.X......................................] duration: 293.400376ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:10:43 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 293.42931ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...........X..............................................] duration: 1.061111432s trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:10:41 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1.06115626s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.284091424s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:10:36 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.284116681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.544006506s trace: https://jaeger.dev.storj.io/trace/3ada5398ef16e7a5 FAIL (2025-11-08 18:10:34 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.544032435s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X....X........X.......................................] duration: 861.6635ms trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:10:29 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 861.691677ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........X...X.............................................] duration: 1.630962584s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:10:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.631017928s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 2.355402112s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:10:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.355434473s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 9 out of 60 is failed (threshold=3) [..X......X.X.X.XX..XX.X.....................................] duration: 287.572979ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:10:16 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 287.602587ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...........X.............................................] duration: 895.572728ms trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:10:13 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 895.602122ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.257724971s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:09:51 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.257752157s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [X..........X...X............................................] duration: 1.746435464s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:09:44 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.746526277s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...........X............................................] duration: 879.575967ms trace: https://jaeger.dev.storj.io/trace/507a0a6af90e29da FAIL (2025-11-08 18:09:41 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 879.604494ms core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 9 out of 60 is failed (threshold=3) [...X......X.X.X.XX..XX.X....................................] duration: 266.817158ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:09:36 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 266.842736ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.476178601s trace: https://jaeger.dev.storj.io/trace/3ada5398ef16e7a5 FAIL (2025-11-08 18:09:35 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.476209422s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X....X........X......................................] duration: 882.956848ms trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:09:20 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 882.985166ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.475523445s trace: https://jaeger.dev.storj.io/trace/667af7ecec748423 FAIL (2025-11-08 18:09:18 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.475548771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 2.55221294s trace: https://jaeger.dev.storj.io/trace/69a842eacf594af2 FAIL (2025-11-08 18:09:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 8 17:40:10.804199015 (221h59m13.505791471s) [upload=195222]: Starting upload Nov 8 17:40:10.804309456 (221h59m13.505901906s) [upload=195222]: Done starting upload Nov 8 17:40:10.804406258 (221h59m13.505998698s) [upload=195222]: Uploading segments... Nov 8 17:40:10.804661289 (221h59m13.506253725s) [upload=195222]: Got next segment. Inline: false Nov 8 17:40:10.804719099 (221h59m13.506311551s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 8 17:40:10.804725438 (221h59m13.506317891s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 8 17:40:10.949670396 (221h59m13.651262854s) [seg_pos={0 0}, upload=195222]: Begin upload segment... Nov 8 17:40:10.949746707 (221h59m13.651339142s) [seg_pos={0 0}, upload=195222]: Done begin upload segment. Nov 8 17:40:10.949790013 (221h59m13.651382473s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950253370 (221h59m13.65184583s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950807476 (221h59m13.652399944s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.950980887 (221h59m13.652573364s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951299826 (221h59m13.652892277s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951606289 (221h59m13.653198938s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.951724371 (221h59m13.653316822s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952145494 (221h59m13.653737946s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952213692 (221h59m13.653806149s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.952860843 (221h59m13.654453304s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.953456202 (221h59m13.655048664s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.954613710 (221h59m13.656206169s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.955958107 (221h59m13.657550571s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.956631592 (221h59m13.658224053s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.958408615 (221h59m13.660001073s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:10.960102983 (221h59m13.661695424s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Uploading piece... Nov 8 17:40:11.554414981 (221h59m14.256007458s) [upload=195222]: Next returned nil segment Nov 8 17:40:11.554435392 (221h59m14.256027861s) [upload=195222]: Waiting for error group managing segments... Nov 8 17:40:11.692643728 (221h59m14.394236176s) [noise=true , address=45.91.116.7:4009 , piece_id=ODWUQB5ECHFVLCNT7ITGFAVWLCSM6LBIVZFY5SKBO4PRSUTMOMZQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.694922910 (221h59m14.396515359s) [noise=true , address=145.79.227.27:4690 , piece_id=AN2BZBRPGNKZJD5QULYN6NQVBA5MAUPNJDPKXRGNJ5JGIE6IWB6Q, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.696239693 (221h59m14.397832128s) [noise=true , address=77.90.31.11:4555 , piece_id=7CLWZCMGVXWS36F2CDJSVLWQRF3FSTRSUGOR7UVAXE6HUJS6V3UQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.704279553 (221h59m14.405872002s) [noise=true , address=145.79.227.20:4437 , piece_id=VSIHFVU3Q7SHJHMUZIVJLHZGOVSXTWH3NAIX4J4ZD65NVVG5V7XQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708695617 (221h59m14.410288057s) [noise=true , address=145.79.227.8:4329 , piece_id=TQPKCTCJY3ELZWXGYPL4SZK2RSRQCACPYBUTID32RHHMOTPMPDAA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.708860281 (221h59m14.410452732s) [noise=true , address=145.79.227.30:4763 , piece_id=POFXWZ7WXWFPMDWOWYRZKL67GHGMU5EZZN43HK5TLIAQ55QVWM4A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.709384760 (221h59m14.410977212s) [noise=true , address=145.79.227.31:4799 , piece_id=J2VT3EXBVJ6MKB2BJWXCAFFXEVM4252KDWEU3OVN3ZQAM4WVZTOA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710418945 (221h59m14.412011393s) [noise=true , address=77.90.31.15:4699 , piece_id=QR3Y5CPZUMRTPJMRPNHVCSMB36GX535CPSAIOARS5IBUODCE5LVA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.710576422 (221h59m14.412168861s) [noise=true , address=145.79.227.2:4114 , piece_id=4MDCN7P2JUQ7S62EJPJ7HDWDJPBWDNAHS4CS6QXJPKHNMDX3BN7A, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713641307 (221h59m14.415233755s) [noise=true , address=45.91.116.12:4226 , piece_id=NSVBQ4V2ATLOSMHQKJEVS5YO63QHGEK2L2WCA5JXCUX3XDNWLZPQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.713802367 (221h59m14.415394815s) [noise=true , address=145.79.227.14:4838 , piece_id=SVTJ5R62XOK4ZKYBOZKWTEHUIB4FZFN7BI7S6C23VRBO6T2MTUAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.715278378 (221h59m14.416870828s) [noise=true , address=45.91.116.7:4010 , piece_id=YDUKICF5PG5BRCEITK2OOI2KYIHZPL6CMQ22SDQ3OC3O3YYIHQGA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718089883 (221h59m14.419682337s) [noise=true , address=45.91.116.12:4227 , piece_id=CHHGEF6WCH5IHGAP4WLHGKFRL7NAHLB5WDCLGXOPBJUWTXYON2FA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: <nil> Nov 8 17:40:11.718111323 (221h59m14.419703784s) [seg_pos={0 0}, upload=195222]: Segment reached optimal threshold of 13 pieces. Nov 8 17:40:11.718414960 (221h59m14.420007418s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718423176 (221h59m14.420015623s) [noise=true , address=145.79.227.13:4803 , piece_id=TIKME7HJKZH3AJQUND4UTFJFHCFBAH7PYGUEBBL4LS2RCD2XSGTQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718551606 (221h59m14.420144063s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718564756 (221h59m14.420157212s) [noise=true , address=145.79.227.22:4510 , piece_id=AMMRIE4NKVKUHOFDKXSWT7JR4DJ75ECODZI7767BVW2ZOBMDNRAQ, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718608200 (221h59m14.420200645s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Nov 8 17:40:11.718617070 (221h59m14.420209524s) [noise=true , address=77.90.31.14:4663 , piece_id=QEGZOKM3V3FDFHDWECT633YDLXRCP3G3NSPRL772UTBBQWTWJ5UA, seg_pos={0 0}, upload=195222]: Long tail cancellation due to optimal threshold Nov 8 17:40:11.718641555 (221h59m14.420234006s) [seg_pos={0 0}, upload=195222]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 8 17:40:11.718729173 (221h59m14.420321626s) [upload=195222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 8 17:40:11.718736122 (221h59m14.420328576s) [upload=195222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 8 17:44:48.551903088 (222h3m51.253495548s) [upload=195222]: Done uploading segments... Nov 8 17:44:48.551995784 (222h3m51.253588254s) [upload=195222]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.55224427s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.629435964s trace: https://jaeger.dev.storj.io/trace/3ada5398ef16e7a5 FAIL (2025-11-08 18:09:06 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.629476913s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X....X........X.....................................] duration: 885.065281ms trace: https://jaeger.dev.storj.io/trace/197ffda43a2a97c2 FAIL (2025-11-08 18:09:01 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 885.186049ms core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 9 out of 60 is failed (threshold=3) [....X......X.X.X.XX..XX.X...................................] duration: 277.894447ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/282d694e86fcb700 FAIL (2025-11-08 18:08:55 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 277.921019ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.X..........X...X...........................................] duration: 1.683533967s trace: https://jaeger.dev.storj.io/trace/6bb9e3317e182100 FAIL (2025-11-08 18:08:50 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 1.683568905s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.506643115s trace: https://jaeger.dev.storj.io/trace/3ada5398ef16e7a5 FAIL (2025-11-08 18:08:39 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.506677675s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)