Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 186407 reports (Page 3551 of 3729)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XX..XX..] duration: 7.005789711s FAIL (2025-07-11 22:07:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.005816352s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XX..XX.] duration: 15.799146717s FAIL (2025-07-11 22:07:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.799169372s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXX.XXX] duration: 30.001494014s FAIL (2025-07-11 22:07:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001510532s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XX..XX] duration: 1m0.119137317s FAIL (2025-07-11 22:06:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Jul 11 22:05:55.321813589 (444h54m55.205227205s) [upload=355785]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:05:55.321818853 (444h54m55.20523248s) [upload=355785]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:05:55.518997859 (444h54m55.402411489s) [seg_pos={0 0}, upload=355785]: Begin upload segment... Jul 11 22:05:55.519056590 (444h54m55.402470217s) [seg_pos={0 0}, upload=355785]: Done begin upload segment. Jul 11 22:05:55.519084972 (444h54m55.402498604s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519101144 (444h54m55.402514779s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519505827 (444h54m55.40291946s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519512227 (444h54m55.402925846s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519800709 (444h54m55.403214338s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.519822072 (444h54m55.403235701s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520126680 (444h54m55.403540312s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520132570 (444h54m55.403546202s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520439936 (444h54m55.403853566s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.520760592 (444h54m55.404174221s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521209831 (444h54m55.404623462s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521511418 (444h54m55.404925045s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.521813719 (444h54m55.405227352s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522154099 (444h54m55.405567732s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522438022 (444h54m55.405851648s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.522736799 (444h54m55.406150429s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.523027776 (444h54m55.406441402s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:55.524220461 (444h54m55.407634093s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Uploading piece... Jul 11 22:05:56.235898282 (444h54m56.119311907s) [upload=355785]: Next returned nil segment Jul 11 22:05:56.235916892 (444h54m56.119330525s) [upload=355785]: Waiting for error group managing segments... Jul 11 22:05:56.412884684 (444h54m56.296298314s) [noise=true , address=79.127.220.98:4431 , piece_id=ISLINB5HRRHMGSW74U4FNH7KP5L3IBJYVWQ4AWGSKUIMC3BOZIKA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.438943831 (444h54m56.322357474s) [noise=true , address=79.127.201.215:4336 , piece_id=HJXWRYIAWYVNUQQUXUN2GQETA3FI7N64JUNJFQ6Y5WR5HI2IGIUA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.460340320 (444h54m56.343753939s) [noise=true , address=79.127.220.101:4064 , piece_id=OQ6SPZNX2HK6HQBHW34VP4UK4HEJYRTBYKOFBRDEA3ON7ET4CN6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:56.780127785 (444h54m56.663541405s) [noise=true , address=64.49.13.27:4965 , piece_id=2RMLKEWRN2VUMCKLPZQYJBYHISY4FG6LWPOLJNK3DDQTWN7OIMQA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.130733263 (444h54m57.01414688s) [noise=true , address=77.90.31.12:4597 , piece_id=LQZFF56AWIDFQHJKQVJ2G5HVFFDPTGVKFVASQKDTASFZNKOLXJGA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:57.194331049 (444h54m57.077744665s) [noise=true , address=77.90.31.16:4768 , piece_id=GY7WDXBTBPNULKIMV7KNWJDQ4YL6HWVOQ7ZX7NRSYO6WXQXM2HJA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.269207970 (444h54m58.152621586s) [noise=true , address=77.90.31.15:4712 , piece_id=252M34NLO35YFQTYC4C7LSIRGSU2MI7VQQGR35EF24GNBX7WAI6A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:05:58.676775084 (444h54m58.560188697s) [noise=true , address=77.90.31.17:4791 , piece_id=P7BWLLYPSLTDZOIBKHSAN4Y2YHJRSCX4XI23W7L7Q7A4GTXJAW2A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:00.880812806 (444h55m0.764226436s) [noise=true , address=77.90.31.11:4562 , piece_id=XZEOCDLDVXX2IURENO6CLVJ6OLM62ZULCCDSH3POS45WY4ZN56LQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.885397124 (444h55m1.768810739s) [noise=true , address=77.90.31.11:4580 , piece_id=KPFBCF3NZXIRYABM3RIXNI4FSRF7S6VWMYTPR4QW6AVTRWA5CICQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:01.908450471 (444h55m1.79186409s) [noise=true , address=45.91.95.2:4035 , piece_id=RU7LOAUF77HCLG7WD6UIAPCBQY4WRS5Z4GZCBIYBPLLIRDNRMRXQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:02.920374484 (444h55m2.8037881s) [noise=true , address=45.91.95.5:4030 , piece_id=KX7ANCBUFRK2UQ4JSZ72FRRSXFJMFOO2JH4LIZBIKMDQ52S7MCTQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:06.990109655 (444h55m6.873523282s) [noise=true , address=45.91.95.3:4022 , piece_id=MIZNO3WDV64YO6EOQFWSPFLLXFLKHFVKRKLKUDQTMKAKQT3ZBOHQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: <nil> Jul 11 22:06:55.050519693 (444h55m54.93393332s) [noise=true , address=104.36.82.20:4463 , piece_id=3KYB27ICW3GAYHZIRQRI3RKI3KKT34IFDMGXN7CTMIBVFLPDI72A, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12vZnQQMq6WQKtsDgDvoE6Qwgp3cPbDSHMZzhCnBuJk1Rx8dT4c, address:104.36.82.20:4463): protocol: write tcp 139.59.144.22:33618->104.36.82.20:4463: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050607213 (444h55m54.934020832s) [noise=true , address=104.36.82.50:4031 , piece_id=2TJ6WQNZZIQBFFAHBXXCKMLRRUR2CGMNJC7UGIP5OMW5TZFNRW5Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: write tcp 139.59.144.22:41868->104.36.82.50:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050629544 (444h55m54.934043166s) [noise=true , address=104.36.82.5:4861 , piece_id=W5WERDTIBDISS6SFK4NWSAD5SM4L5EUMTSGX5R3PH7EB3K7GGOLA, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:104.36.82.5:4861): protocol: write tcp 139.59.144.22:50224->104.36.82.5:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050646322 (444h55m54.934059943s) [noise=true , address=104.36.82.4:4824 , piece_id=E5J5EF6P2RFCSKMGY3CSPB4ZT2SIF2R7QXCJV6ZRIU2P6YUTAI6Q, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:128RwAx77X1bmYyNBtGGMbUUKtvpZjtSBRK2M7AKtebEP4TY4wm, address:104.36.82.4:4824): protocol: rpc: dial tcp 104.36.82.4:4824: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050671506 (444h55m54.934085133s) [noise=true , address=104.36.82.22:4535 , piece_id=FWOYPFPCNITCDZL5TYIU4W6IJZ7PCEWDKHC3IBJ37TLVSPVC67QQ, seg_pos={0 0}, upload=355785]: Done uploading piece. err: ecclient: upload failed (node:1GEmVgDhSMyFLPY5qcc4gGqD9amUzhZ2iduyWusUSrVjLgBu1T, address:104.36.82.22:4535): protocol: write tcp 139.59.144.22:34212->104.36.82.22:4535: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:55.050733238 (444h55m54.934146868s) [seg_pos={0 0}, upload=355785]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:06:55.167737334 (444h55m55.051150954s) [upload=355785]: Done uploading segments... Jul 11 22:06:55.167778274 (444h55m55.051191901s) [upload=355785]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.119155832s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXX.XX] duration: 30.105881644s FAIL (2025-07-11 22:06:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== UE7N7RBTDETKFO2VBQHMTQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.133666060 (444h55m5.017079696s) [noise=true , address=104.36.82.9:4005 , piece_id=PPFUSA2W6GEWONBYBRT52YZEURQATGSDVERB7NW5FI4TEGLO33ZA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134087567 (444h55m5.017501209s) [noise=true , address=104.36.82.13:4187 , piece_id=64IFFQLSGZL6K2Z5V4J4COODY7KXKHVOQOEGOZZBDJW3RGWJDSEA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134256035 (444h55m5.017669668s) [noise=true , address=77.90.31.17:4772 , piece_id=4UHYRDGZHM7FPD7ZK46SFXI54HRMNBWISG53OJWVLPPIJHG7PUDQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134495636 (444h55m5.017909288s) [noise=true , address=104.36.82.3:4767 , piece_id=KUQBH2WEMXX6Y42ILGHDNIEKE6OVTX2F3XK5X4IHXTRODLILJDSQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.134897654 (444h55m5.018311287s) [noise=true , address=104.36.82.10:4078 , piece_id=5MVTGTPKB3IOV4YVQB4KGMC7JKQRGYIBKCHP4LKGLORG34HYJDIQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135292900 (444h55m5.018706535s) [noise=true , address=104.36.82.15:4258 , piece_id=LAS3CRJZDRQZLMOF22ETEVZ7O67NJJE4YE2UB7RFY5IDDRGWYQ6A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135552707 (444h55m5.018966337s) [noise=true , address=104.36.82.12:4151 , piece_id=4YDLARZNKEKGHKZXNDNM3JBNA3CPADNSTD7UAMDUU4BANDGIUDFQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.135900233 (444h55m5.01931386s) [noise=true , address=77.90.31.13:4626 , piece_id=OUT476GEFNOWZWOJXRFFBZY6TJSUYSE6ZBZFJXQAC4OKVGBGUWOQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136252343 (444h55m5.019665975s) [noise=true , address=104.36.82.11:4115 , piece_id=GH6QVTPGIF554M7RTL3CJGTVB4SS444DVNGJWVCGJ6XJKVQ6N2OQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136498346 (444h55m5.019911992s) [noise=true , address=77.90.31.11:4554 , piece_id=RF3JPVM74Q5BT4XNKZLPASES7ZRMN75THDXVIHZTUWZPHQI7R2BA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.136900343 (444h55m5.020313979s) [noise=true , address=104.36.82.7:4911 , piece_id=ONQAL7WGOQZFGCWGR3DTWRSTWAX2TPM2YFLHMSMO3RHU3HSVYA3A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137291378 (444h55m5.020705006s) [noise=true , address=45.91.116.4:4939 , piece_id=FN2DHKEHLNHDJRYJPY73D447FCQLFCNSSAGES3DYXRI4ISXJSYZA, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137399682 (444h55m5.020813312s) [noise=true , address=104.36.82.6:4876 , piece_id=RLEFABSMYPGIUS4JHOTJL6GZGEOPPJLV3ZRA5CZKCBPYEO5ZQRAQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.137712035 (444h55m5.021125668s) [noise=true , address=104.36.82.16:4295 , piece_id=CNHBOFILZ2EFPQI5R4VJJYK2L4YNX4B6RKXI73OOZUYKRZBM2L6A, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:05.138076815 (444h55m5.021490442s) [noise=true , address=77.90.31.10:4518 , piece_id=MJDPMI2GFHTNIZ24CDPDPBCK6GZ3LEKRZCU4FAPMDA7ZUK5EW3AQ, seg_pos={0 0}, upload=355787]: Uploading piece... Jul 11 22:06:06.709355010 (444h55m6.59276863s) [upload=355787]: Next returned nil segment Jul 11 22:06:06.709388206 (444h55m6.592801833s) [upload=355787]: Waiting for error group managing segments... Jul 11 22:06:08.096853372 (444h55m7.980267047s) [noise=true , address=77.90.31.11:4554 , piece_id=RF3JPVM74Q5BT4XNKZLPASES7ZRMN75THDXVIHZTUWZPHQI7R2BA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:10.196362260 (444h55m10.079775872s) [noise=true , address=77.90.31.13:4626 , piece_id=OUT476GEFNOWZWOJXRFFBZY6TJSUYSE6ZBZFJXQAC4OKVGBGUWOQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:15.218285149 (444h55m15.101698768s) [noise=true , address=77.90.31.10:4518 , piece_id=MJDPMI2GFHTNIZ24CDPDPBCK6GZ3LEKRZCU4FAPMDA7ZUK5EW3AQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:16.129501415 (444h55m16.012915049s) [noise=true , address=77.90.31.17:4772 , piece_id=4UHYRDGZHM7FPD7ZK46SFXI54HRMNBWISG53OJWVLPPIJHG7PUDQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:17.385731005 (444h55m17.26914464s) [noise=true , address=45.91.116.4:4939 , piece_id=FN2DHKEHLNHDJRYJPY73D447FCQLFCNSSAGES3DYXRI4ISXJSYZA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: <nil> Jul 11 22:06:34.749342136 (444h55m34.632755751s) [noise=true , address=104.36.82.3:4767 , piece_id=KUQBH2WEMXX6Y42ILGHDNIEKE6OVTX2F3XK5X4IHXTRODLILJDSQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749436984 (444h55m34.632850611s) [noise=true , address=104.36.82.12:4151 , piece_id=4YDLARZNKEKGHKZXNDNM3JBNA3CPADNSTD7UAMDUU4BANDGIUDFQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749543804 (444h55m34.632957422s) [noise=true , address=104.36.82.13:4187 , piece_id=64IFFQLSGZL6K2Z5V4J4COODY7KXKHVOQOEGOZZBDJW3RGWJDSEA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: write tcp 139.59.144.22:37398->104.36.82.13:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749610349 (444h55m34.633023965s) [noise=true , address=104.36.82.7:4911 , piece_id=ONQAL7WGOQZFGCWGR3DTWRSTWAX2TPM2YFLHMSMO3RHU3HSVYA3A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: rpc: dial tcp 104.36.82.7:4911: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749661721 (444h55m34.633075338s) [noise=true , address=104.36.82.6:4876 , piece_id=RLEFABSMYPGIUS4JHOTJL6GZGEOPPJLV3ZRA5CZKCBPYEO5ZQRAQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:41804->104.36.82.6:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749712162 (444h55m34.633125778s) [noise=true , address=104.36.82.14:4221 , piece_id=K4I4QC4YKAW6M5UXVLNKTPCKEI6NQNUE7N7RBTDETKFO2VBQHMTQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 139.59.144.22:41990->104.36.82.14:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749863225 (444h55m34.633276845s) [noise=true , address=104.36.82.9:4005 , piece_id=PPFUSA2W6GEWONBYBRT52YZEURQATGSDVERB7NW5FI4TEGLO33ZA, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:48210->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.749996307 (444h55m34.633409933s) [noise=true , address=104.36.82.10:4078 , piece_id=5MVTGTPKB3IOV4YVQB4KGMC7JKQRGYIBKCHP4LKGLORG34HYJDIQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750038894 (444h55m34.63345251s) [noise=true , address=104.36.82.16:4295 , piece_id=CNHBOFILZ2EFPQI5R4VJJYK2L4YNX4B6RKXI73OOZUYKRZBM2L6A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 139.59.144.22:49444->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750048359 (444h55m34.633461975s) [noise=true , address=104.36.82.11:4115 , piece_id=GH6QVTPGIF554M7RTL3CJGTVB4SS444DVNGJWVCGJ6XJKVQ6N2OQ, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: write tcp 139.59.144.22:38672->104.36.82.11:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750091039 (444h55m34.633504657s) [noise=true , address=104.36.82.15:4258 , piece_id=LAS3CRJZDRQZLMOF22ETEVZ7O67NJJE4YE2UB7RFY5IDDRGWYQ6A, seg_pos={0 0}, upload=355787]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:47840->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:06:34.750126727 (444h55m34.633540353s) [seg_pos={0 0}, upload=355787]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 11 22:06:34.853461811 (444h55m34.736875429s) [upload=355787]: Done uploading segments... Jul 11 22:06:34.853512446 (444h55m34.736926063s) [upload=355787]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.105909107s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XXX.XX] duration: 30.000833441s FAIL (2025-07-11 22:06:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000860292s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX..X] duration: 1m0.106845319s FAIL (2025-07-11 22:05:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== .. Jul 11 22:04:55.072689878 (444h53m54.956103505s) [upload=355771]: Got next segment. Inline: false Jul 11 22:04:55.072753570 (444h53m54.956167196s) [upload=355771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:04:55.072759932 (444h53m54.956173558s) [upload=355771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:04:55.313585304 (444h53m55.19699899s) [seg_pos={0 0}, upload=355771]: Begin upload segment... Jul 11 22:04:55.313726244 (444h53m55.197139859s) [noise=true , address=77.90.31.15:4730 , piece_id=APUA6M3ZLFKYXNUUU2SOQ4MUA7YVDEGWN6H6FXV5NWCJSJFMPSBA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.313758278 (444h53m55.197171913s) [noise=true , address=104.36.82.39:4735 , piece_id=6TN4HDENV6FZXRU6MXF623TVZUSBB6NJAA2JWDNFLV3CF565XG2A, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314133360 (444h53m55.197546995s) [noise=true , address=45.91.116.5:4304 , piece_id=LVPWMZCOTZQRCUYFWVTRA5BWKVRDHS2ZUNZTJKJGWACUWOKM3SCQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314369395 (444h53m55.197783007s) [seg_pos={0 0}, upload=355771]: Done begin upload segment. Jul 11 22:04:55.314377578 (444h53m55.197791191s) [noise=true , address=45.91.95.2:4018 , piece_id=QH5M55Q2R6OC4IHELLZQRZXTPQK5MIOBJOXQ3MATWRIUEZPP4HMA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314387799 (444h53m55.197801426s) [noise=true , address=45.91.95.3:4004 , piece_id=5PMG6JYVOSW2FT2Y47QL2E5YVI3A54TV65SMHMQMG3OQX5XD5U5Q, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.314696651 (444h53m55.198110279s) [noise=true , address=77.90.31.17:4769 , piece_id=XVHHMBMH3SYRXDLTSRUSE3OUKHYEH5QNU4Y6PQGDQXUT5FXK4TNA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.315247269 (444h53m55.198660893s) [noise=true , address=77.90.31.13:4629 , piece_id=LDXOLEO34KPI5LG5FSNAOIWV2K5AJOXGNRILNJJOO4KQZ6QBU3SQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.318188723 (444h53m55.201602358s) [noise=true , address=79.127.219.38:4035 , piece_id=IOLSZGLRTYUA37HWFGCFO4CXQATUHBLTCE2HPVNDDJRAQ3L3KFPA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.318647317 (444h53m55.202060946s) [noise=true , address=77.90.31.19:4853 , piece_id=7O7DDEURX4VA5JBNETWGAJWGT244MFNTPMNJG33KJFOHWYWUSVAA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.319019338 (444h53m55.202432967s) [noise=true , address=79.127.205.234:4598 , piece_id=HDCUJRXFWHSGNYBXM67BQESYXHTHBLRT2E2SH43U6SKBBXZ54EKA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.319406715 (444h53m55.202820351s) [noise=true , address=104.36.82.14:4227 , piece_id=XGVFZPSGVK2OVWY7HJC2AHISRZBSYEM6FFZGHBP5OLX5HU44NGTA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.320615595 (444h53m55.204029249s) [noise=true , address=104.36.82.51:4053 , piece_id=4UYZRIMBWYUOA42PJ2TLAO2IOGIIXEWCBDNQWCPZOHKKOFJKCAQA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.321201069 (444h53m55.204614709s) [noise=true , address=104.36.82.18:4396 , piece_id=TQ37GIZZ4DBLH2XVNWQV4QXY4K3AJ5NXFP6RZJ32OZKAASPNEVLA, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.321816892 (444h53m55.205230528s) [noise=true , address=45.91.95.4:4008 , piece_id=QCG67FGPEPSYXK22CLKPQRQXZCPEB5PI2M4MOQYZLKYR4C33WPGQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.322188584 (444h53m55.205602214s) [noise=true , address=104.36.82.40:4965 , piece_id=2KFI3M4DBLMBXNALUZG3X2GBXLD3RHDOYPEX4E72WGP5D43YR46Q, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.322553656 (444h53m55.205967285s) [noise=true , address=45.91.116.12:4252 , piece_id=JGSTDTCUZ53KDL4MEHNZ5NMMNHV6IG7JVFQNRA76O5G6BMFMNUCQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.323220751 (444h53m55.206634384s) [noise=true , address=77.90.31.18:4811 , piece_id=7CJCDEFU4523D4YHMSFQPA7STYFYUG4VZQRPMICNCSBULG6DLRWQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.323528348 (444h53m55.206941974s) [noise=true , address=77.90.31.12:4605 , piece_id=K7AUJNYTUNSRA6UBDPQI2DF3QEI4X3HKG6ANOK5NAUXTGJRHJEDQ, seg_pos={0 0}, upload=355771]: Uploading piece... Jul 11 22:04:55.902170512 (444h53m55.785584152s) [upload=355771]: Next returned nil segment Jul 11 22:04:55.902218409 (444h53m55.785632036s) [upload=355771]: Waiting for error group managing segments... Jul 11 22:04:56.598198976 (444h53m56.4816126s) [noise=true , address=79.127.205.234:4598 , piece_id=HDCUJRXFWHSGNYBXM67BQESYXHTHBLRT2E2SH43U6SKBBXZ54EKA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:57.441463597 (444h53m57.324877217s) [noise=true , address=77.90.31.15:4730 , piece_id=APUA6M3ZLFKYXNUUU2SOQ4MUA7YVDEGWN6H6FXV5NWCJSJFMPSBA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:57.617801779 (444h53m57.501215406s) [noise=true , address=45.91.116.5:4304 , piece_id=LVPWMZCOTZQRCUYFWVTRA5BWKVRDHS2ZUNZTJKJGWACUWOKM3SCQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:59.206776760 (444h53m59.090190379s) [noise=true , address=77.90.31.17:4769 , piece_id=XVHHMBMH3SYRXDLTSRUSE3OUKHYEH5QNU4Y6PQGDQXUT5FXK4TNA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:04:59.806024373 (444h53m59.689438009s) [noise=true , address=77.90.31.18:4811 , piece_id=7CJCDEFU4523D4YHMSFQPA7STYFYUG4VZQRPMICNCSBULG6DLRWQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:00.604640851 (444h54m0.488054478s) [noise=true , address=77.90.31.19:4853 , piece_id=7O7DDEURX4VA5JBNETWGAJWGT244MFNTPMNJG33KJFOHWYWUSVAA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:02.621235083 (444h54m2.5046487s) [noise=true , address=79.127.219.38:4035 , piece_id=IOLSZGLRTYUA37HWFGCFO4CXQATUHBLTCE2HPVNDDJRAQ3L3KFPA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:04.045177388 (444h54m3.92859102s) [noise=true , address=77.90.31.12:4605 , piece_id=K7AUJNYTUNSRA6UBDPQI2DF3QEI4X3HKG6ANOK5NAUXTGJRHJEDQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:04.409180174 (444h54m4.292593788s) [noise=true , address=45.91.116.12:4252 , piece_id=JGSTDTCUZ53KDL4MEHNZ5NMMNHV6IG7JVFQNRA76O5G6BMFMNUCQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:07.797191258 (444h54m7.680604874s) [noise=true , address=77.90.31.13:4629 , piece_id=LDXOLEO34KPI5LG5FSNAOIWV2K5AJOXGNRILNJJOO4KQZ6QBU3SQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:11.085763116 (444h54m10.969176745s) [noise=true , address=45.91.95.3:4004 , piece_id=5PMG6JYVOSW2FT2Y47QL2E5YVI3A54TV65SMHMQMG3OQX5XD5U5Q, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:28.810582170 (444h54m28.693995785s) [noise=true , address=45.91.95.2:4018 , piece_id=QH5M55Q2R6OC4IHELLZQRZXTPQK5MIOBJOXQ3MATWRIUEZPP4HMA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:51.180763268 (444h54m51.064176887s) [noise=true , address=45.91.95.4:4008 , piece_id=QCG67FGPEPSYXK22CLKPQRQXZCPEB5PI2M4MOQYZLKYR4C33WPGQ, seg_pos={0 0}, upload=355771]: Done uploading piece. err: <nil> Jul 11 22:05:54.802082913 (444h54m54.685496543s) [noise=true , address=104.36.82.39:4735 , piece_id=6TN4HDENV6FZXRU6MXF623TVZUSBB6NJAA2JWDNFLV3CF565XG2A, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1QCQZ8eFGLa5TAaAJJzUip8o5DAELzTBTm4UFbt3Am5yYWXAZF, address:104.36.82.39:4735): protocol: rpc: dial tcp 104.36.82.39:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802160234 (444h54m54.685573849s) [noise=true , address=104.36.82.51:4053 , piece_id=4UYZRIMBWYUOA42PJ2TLAO2IOGIIXEWCBDNQWCPZOHKKOFJKCAQA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:12DqyTc4MHvxutFXs2WWRCk8MSuxF3Y1HayyQY4Y5sy8cqDLWuG, address:104.36.82.51:4053): protocol: rpc: dial tcp 104.36.82.51:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802233684 (444h54m54.685647315s) [noise=true , address=104.36.82.40:4965 , piece_id=2KFI3M4DBLMBXNALUZG3X2GBXLD3RHDOYPEX4E72WGP5D43YR46Q, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1ToFTqjRAWvpRQzf6vyHKDpforPyb4LbDGcCVhRhkwiBGKihnD, address:104.36.82.40:4965): protocol: rpc: dial tcp 104.36.82.40:4965: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802337037 (444h54m54.685750651s) [noise=true , address=104.36.82.14:4227 , piece_id=XGVFZPSGVK2OVWY7HJC2AHISRZBSYEM6FFZGHBP5OLX5HU44NGTA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:104.36.82.14:4227): protocol: rpc: dial tcp 104.36.82.14:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:54.802419137 (444h54m54.685832764s) [noise=true , address=104.36.82.18:4396 , piece_id=TQ37GIZZ4DBLH2XVNWQV4QXY4K3AJ5NXFP6RZJ32OZKAASPNEVLA, seg_pos={0 0}, upload=355771]: Done uploading piece. err: ecclient: upload failed (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx, address:104.36.82.18:4396): protocol: write tcp 139.59.144.22:57994->104.36.82.18:4396: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 11 22:05:54.802441878 (444h54m54.685855512s) [seg_pos={0 0}, upload=355771]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:05:54.906760648 (444h54m54.790174283s) [upload=355771]: Done uploading segments... Jul 11 22:05:54.906793413 (444h54m54.790207044s) [upload=355771]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.106867521s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXX.X] duration: 30.109352013s FAIL (2025-07-11 22:05:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== B3TRY7DESLM7RITRW3N25NU6RSLNMS53YTKG5M3ICYHL6A, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.648097583 (444h53m59.531511213s) [noise=true , address=104.36.82.16:4295 , piece_id=HAEASJ6R5JE3C7KYMP4VQ5TZEKOF33XADVXFEBAC3XNVI2EZY2YA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.648622050 (444h53m59.532035684s) [noise=true , address=77.90.31.19:4843 , piece_id=6TXPOBW44VUWY2HU33TZ4VILNJJORJ5UCNCRCDVVOH3RCC3RXVGA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649033035 (444h53m59.532446676s) [noise=true , address=104.36.82.15:4258 , piece_id=JXFOVVJSVLUOI3XMRAWICOKUVSLGPSNZPT7S3RZFSCZQP74GM7VA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649060162 (444h53m59.532473787s) [noise=true , address=45.91.116.13:4153 , piece_id=CUHA4TL64SWHYZ6XU5C7RGUPABM4N26YVEOO6NA7PS6ZN4ZLWFWQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649485981 (444h53m59.532899616s) [noise=true , address=77.90.31.18:4807 , piece_id=O5VMFRIB776MOJSNCGQ56NJDBNTA37CUECNLU766TAIRK766XS4Q, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649688620 (444h53m59.533102257s) [noise=true , address=104.36.82.10:4078 , piece_id=6FXQSLTPKPLXHKUKRWTP2SVOCDGLG6VHQRQ7KASKOQOJSS4746XA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.649855897 (444h53m59.533269532s) [noise=true , address=104.36.82.2:4732 , piece_id=UXLFRQF2NNCI4DGQYMVN5L3W7OXLUGA2MZOUTY3C7KKZFKUM4B4Q, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650270507 (444h53m59.533684143s) [noise=true , address=104.36.82.9:4007 , piece_id=IA6UICMJEFAPRL7IUG5FEOB2YODPXR6UZEYCKZIXDAVBNVCB4PZA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650278311 (444h53m59.533691947s) [noise=true , address=104.36.82.6:4875 , piece_id=U3T7QIIHRHPVY2RN3GMUL66JCWU5AYFFU4OWJBOGZB2SFVBHR7GA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650316741 (444h53m59.533730353s) [noise=true , address=104.36.82.5:4840 , piece_id=FCU7E7ZA3KGTHYB3ZRW6SQAKYMMLLJYE7FROIJCRXIBW6EADUC3A, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.650974066 (444h53m59.534387741s) [noise=true , address=77.90.31.14:4663 , piece_id=AIFMRAEYZ2UBGGT5BBNP5RS2SQWUGTVWPE76J2H3HT4KY2ELBIKA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.651716428 (444h53m59.535130055s) [noise=true , address=104.36.82.8:4969 , piece_id=ZB7AL22XVG32DFXJFUETTJ7TICHM5FTRZLFWU2AYOLCBZNISUWPA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.652630735 (444h53m59.536044373s) [noise=true , address=45.91.116.7:4010 , piece_id=3WSD6MQW6G6CC2VKYGKFT7YEVE47X6OQAZBXTORDHEBTJ42SHUJQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.653573814 (444h53m59.536987442s) [noise=true , address=104.36.82.14:4223 , piece_id=RJ6HCZ5HCCFUQMBUP7KM5GMX24X2Y4VM3ADELWR6I5GGPAUO24QQ, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:04:59.654426947 (444h53m59.537840589s) [noise=true , address=104.36.82.16:4293 , piece_id=F4BALD5B6J4ITDPZ62AURSSNNGKWATJVPFC7BC7WDNS5SB2KIQBA, seg_pos={0 0}, upload=355774]: Uploading piece... Jul 11 22:05:06.634724509 (444h54m6.518138135s) [upload=355774]: Next returned nil segment Jul 11 22:05:06.634746228 (444h54m6.518159866s) [upload=355774]: Waiting for error group managing segments... Jul 11 22:05:12.832681435 (444h54m12.716095056s) [noise=true , address=77.90.31.14:4663 , piece_id=AIFMRAEYZ2UBGGT5BBNP5RS2SQWUGTVWPE76J2H3HT4KY2ELBIKA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:13.441666250 (444h54m13.325079877s) [noise=true , address=77.90.31.18:4807 , piece_id=O5VMFRIB776MOJSNCGQ56NJDBNTA37CUECNLU766TAIRK766XS4Q, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:13.629841060 (444h54m13.51325469s) [noise=true , address=45.91.116.13:4153 , piece_id=CUHA4TL64SWHYZ6XU5C7RGUPABM4N26YVEOO6NA7PS6ZN4ZLWFWQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:20.797771223 (444h54m20.681184839s) [noise=true , address=45.91.116.7:4010 , piece_id=3WSD6MQW6G6CC2VKYGKFT7YEVE47X6OQAZBXTORDHEBTJ42SHUJQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:22.776372688 (444h54m22.659786304s) [noise=true , address=77.90.31.19:4843 , piece_id=6TXPOBW44VUWY2HU33TZ4VILNJJORJ5UCNCRCDVVOH3RCC3RXVGA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: <nil> Jul 11 22:05:29.256798257 (444h54m29.140211877s) [noise=true , address=104.36.82.10:4078 , piece_id=6FXQSLTPKPLXHKUKRWTP2SVOCDGLG6VHQRQ7KASKOQOJSS4746XA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.256931538 (444h54m29.140345156s) [noise=true , address=104.36.82.2:4732 , piece_id=UXLFRQF2NNCI4DGQYMVN5L3W7OXLUGA2MZOUTY3C7KKZFKUM4B4Q, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 139.59.144.22:58868->104.36.82.2:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257021976 (444h54m29.140435603s) [noise=true , address=104.36.82.9:4007 , piece_id=IA6UICMJEFAPRL7IUG5FEOB2YODPXR6UZEYCKZIXDAVBNVCB4PZA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257128785 (444h54m29.140542399s) [noise=true , address=104.36.82.3:4767 , piece_id=53FPKVB3TRY7DESLM7RITRW3N25NU6RSLNMS53YTKG5M3ICYHL6A, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:60582->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257150641 (444h54m29.140564253s) [noise=true , address=104.36.82.15:4258 , piece_id=JXFOVVJSVLUOI3XMRAWICOKUVSLGPSNZPT7S3RZFSCZQP74GM7VA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:50642->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257211975 (444h54m29.140625607s) [noise=true , address=104.36.82.16:4295 , piece_id=HAEASJ6R5JE3C7KYMP4VQ5TZEKOF33XADVXFEBAC3XNVI2EZY2YA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257305598 (444h54m29.140719227s) [noise=true , address=104.36.82.6:4875 , piece_id=U3T7QIIHRHPVY2RN3GMUL66JCWU5AYFFU4OWJBOGZB2SFVBHR7GA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: rpc: dial tcp 104.36.82.6:4875: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257365719 (444h54m29.140779347s) [noise=true , address=104.36.82.8:4969 , piece_id=ZB7AL22XVG32DFXJFUETTJ7TICHM5FTRZLFWU2AYOLCBZNISUWPA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: rpc: dial tcp 104.36.82.8:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257479175 (444h54m29.140892803s) [noise=true , address=104.36.82.14:4223 , piece_id=RJ6HCZ5HCCFUQMBUP7KM5GMX24X2Y4VM3ADELWR6I5GGPAUO24QQ, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:52210->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257571361 (444h54m29.140984974s) [noise=true , address=104.36.82.16:4293 , piece_id=F4BALD5B6J4ITDPZ62AURSSNNGKWATJVPFC7BC7WDNS5SB2KIQBA, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:51066->104.36.82.16:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257632322 (444h54m29.141045949s) [noise=true , address=104.36.82.5:4840 , piece_id=FCU7E7ZA3KGTHYB3ZRW6SQAKYMMLLJYE7FROIJCRXIBW6EADUC3A, seg_pos={0 0}, upload=355774]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: write tcp 139.59.144.22:46436->104.36.82.5:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:05:29.257655258 (444h54m29.141068894s) [seg_pos={0 0}, upload=355774]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 11 22:05:29.363237589 (444h54m29.246651207s) [upload=355774]: Done uploading segments... Jul 11 22:05:29.363263156 (444h54m29.246676782s) [upload=355774]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.109448291s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 18.089470142s FAIL (2025-07-11 22:05:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.08952218s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X......................................................XXX.X] duration: 30.001510929s FAIL (2025-07-11 22:05:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001900675s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 12.63924171s FAIL (2025-07-11 22:05:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 12.639268864s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 9.236778677s FAIL (2025-07-11 22:04:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 737s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.488709574 (444h51m30.372123209s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489246417 (444h51m30.372660054s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489303194 (444h51m30.372716825s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489648268 (444h51m30.373061904s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489707285 (444h51m30.373120911s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490095044 (444h51m30.373508684s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490197063 (444h51m30.373610693s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490604300 (444h51m30.374017929s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491042169 (444h51m30.374455796s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491492265 (444h51m30.374905902s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491866085 (444h51m30.375279717s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492126748 (444h51m30.375540385s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492565072 (444h51m30.375978713s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493333026 (444h51m30.376746662s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493702567 (444h51m30.377116204s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494079587 (444h51m30.377493219s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494555851 (444h51m30.377969484s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:31.011808775 (444h51m30.895222393s) [upload=355744]: Next returned nil segment Jul 11 22:02:31.011828319 (444h51m30.895241945s) [upload=355744]: Waiting for error group managing segments... Jul 11 22:02:31.185708662 (444h51m31.069122287s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.203802874 (444h51m31.087216495s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.205879153 (444h51m31.089292766s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.219680112 (444h51m31.103093739s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.222479897 (444h51m31.105893526s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230662544 (444h51m31.114076159s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230778357 (444h51m31.114191976s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.334409626 (444h51m31.217823263s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.339502902 (444h51m31.222916533s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.388877737 (444h51m31.272291372s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389126192 (444h51m31.272539822s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389800397 (444h51m31.273214027s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.426833520 (444h51m31.31024715s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:03:30.023868435 (444h52m29.907282053s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:30.024114203 (444h52m29.90752783s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1pbWgKsDigGSwnWtNHwsJC3srt45YiUhswf2hv9gbn2LSCLkSH, address:104.36.82.18:4374): protocol: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection Jul 11 22:03:30.024186106 (444h52m29.907599732s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:13va7LjDtQV2ob7mrUaJUXc64HmZG3svoDaErmypjYHVAcw1o3, address:104.36.82.40:4962): protocol: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection Jul 11 22:03:30.024207244 (444h52m29.907620871s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:104.36.82.34:4551): protocol: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection Jul 11 22:03:30.024253975 (444h52m29.907667602s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1R66QktZdYxwpsjALURfq9eEvqNG2tfMvBjC5DnackVppZdGpT, address:104.36.82.17:4336): protocol: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection Jul 11 22:03:30.024291452 (444h52m29.907705088s) [seg_pos={0 0}, upload=355744]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:30.137783063 (444h52m30.021196699s) [upload=355744]: Done uploading segments... Jul 11 22:03:30.137819589 (444h52m30.021233226s) [upload=355744]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 9.236809975s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 8.55313247s FAIL (2025-07-11 22:04:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224183415 (444h52m26.107597051s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224517315 (444h52m26.107930946s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224812761 (444h52m26.10822639s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225181978 (444h52m26.108595613s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225508695 (444h52m26.108922325s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225792461 (444h52m26.109206097s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226172056 (444h52m26.109585688s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226481110 (444h52m26.10989474s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226831712 (444h52m26.110245341s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.227131111 (444h52m26.11054474s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:27.302994623 (444h52m27.186408257s) [upload=355755]: Next returned nil segment Jul 11 22:03:27.303018528 (444h52m27.186432164s) [upload=355755]: Waiting for error group managing segments... Jul 11 22:03:27.876648904 (444h52m27.760062523s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:28.609717180 (444h52m28.493130792s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.013638539 (444h52m30.897052169s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.108297749 (444h52m30.991711367s) [noise=true , address=45.91.116.14:4190 , piece_id=TNIBGC6OIB5KS325JUH35L2YJYFOO4HD4XNOQX55TKQR5CRYUUHQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:33.471534051 (444h52m33.35494767s) [noise=true , address=45.91.116.11:4117 , piece_id=DIDIAZVBKR3SEC7QANMXIR6XUGWXDQQUHM6WA7XTHEYF7OLZJIYQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:37.297262395 (444h52m37.180676021s) [noise=true , address=45.91.116.14:4189 , piece_id=S2N5INQUI4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:55.824710938 (444h52m55.708124551s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:55.825086456 (444h52m55.708500072s) [noise=true , address=104.36.82.9:4006 , piece_id=4FGZSUZ54DHILWD7SFWNRXKQJJG6UKRTKCYYAOSYY5Q4IVTPW6JA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection Jul 11 22:03:55.825165541 (444h52m55.708579153s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection Jul 11 22:03:55.825228546 (444h52m55.708642158s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection Jul 11 22:03:55.825237626 (444h52m55.708651277s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection Jul 11 22:03:55.825297732 (444h52m55.708711344s) [noise=true , address=104.36.82.16:4293 , piece_id=VCINGGXB7VIPIQTGNM5II2EISTTY32OACYGGQZZ7VKKZZZHXBPTA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection Jul 11 22:03:55.825363916 (444h52m55.708777529s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection Jul 11 22:03:55.825421328 (444h52m55.708834939s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection Jul 11 22:03:55.825463724 (444h52m55.708877335s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection Jul 11 22:03:55.825477966 (444h52m55.708891585s) [noise=true , address=104.36.82.12:4149 , piece_id=UNAE4IGDADDPLGWRJW6S3JC5AFNSH44REOGKBTV3F7G65YY7EQDQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection Jul 11 22:03:55.825533457 (444h52m55.708947087s) [seg_pos={0 0}, upload=355755]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:03:55.935680781 (444h52m55.819094399s) [upload=355755]: Done uploading segments... Jul 11 22:03:55.935707519 (444h52m55.819121146s) [upload=355755]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 8.553163356s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XX......................................................XXX.] duration: 5.451629743s FAIL (2025-07-11 22:04:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.451650108s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.112466088s FAIL (2025-07-11 22:03:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224183415 (444h52m26.107597051s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224517315 (444h52m26.107930946s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.224812761 (444h52m26.10822639s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225181978 (444h52m26.108595613s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225508695 (444h52m26.108922325s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.225792461 (444h52m26.109206097s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226172056 (444h52m26.109585688s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226481110 (444h52m26.10989474s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.226831712 (444h52m26.110245341s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:26.227131111 (444h52m26.11054474s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Uploading piece... Jul 11 22:03:27.302994623 (444h52m27.186408257s) [upload=355755]: Next returned nil segment Jul 11 22:03:27.303018528 (444h52m27.186432164s) [upload=355755]: Waiting for error group managing segments... Jul 11 22:03:27.876648904 (444h52m27.760062523s) [noise=true , address=77.90.31.18:4806 , piece_id=LFNM3KVOSZMB2ZOO23KEDHLP5ZY3XJLF7ZINRWGV6CURDXHEH35Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:28.609717180 (444h52m28.493130792s) [noise=true , address=45.91.116.4:4939 , piece_id=7R4FHPHRZAPYV6OXPKA7IDVLCOVWZAZMAXKHBVY2NL57OE2QCZ4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.013638539 (444h52m30.897052169s) [noise=true , address=77.90.31.14:4663 , piece_id=SH7HEPZEDQ4RX7IQMECDVTNIVIKT2NX75IPZDLQHCEFONH3MXB4Q, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:31.108297749 (444h52m30.991711367s) [noise=true , address=45.91.116.14:4190 , piece_id=TNIBGC6OIB5KS325JUH35L2YJYFOO4HD4XNOQX55TKQR5CRYUUHQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:33.471534051 (444h52m33.35494767s) [noise=true , address=45.91.116.11:4117 , piece_id=DIDIAZVBKR3SEC7QANMXIR6XUGWXDQQUHM6WA7XTHEYF7OLZJIYQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:37.297262395 (444h52m37.180676021s) [noise=true , address=45.91.116.14:4189 , piece_id=S2N5INQUI4KA2CWK37V5LCFHGRPYKVBDXRYQZ4LUTMD6ZBGLELUQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: <nil> Jul 11 22:03:55.824710938 (444h52m55.708124551s) [noise=true , address=104.36.82.8:4971 , piece_id=EIVPU6EWWKJSRYSMWACCY6SMFD27KFBG7DNWFD6M7WCJROUWIMIQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:55.825086456 (444h52m55.708500072s) [noise=true , address=104.36.82.9:4006 , piece_id=4FGZSUZ54DHILWD7SFWNRXKQJJG6UKRTKCYYAOSYY5Q4IVTPW6JA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:35766->104.36.82.9:4006: use of closed network connection Jul 11 22:03:55.825165541 (444h52m55.708579153s) [noise=true , address=104.36.82.15:4259 , piece_id=UXFXYHBT24C3U25JZVCOEKLBPU5VRGCPX3SDNRJCHTHKLEAULUZA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:104.36.82.15:4259): protocol: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59176->104.36.82.15:4259: use of closed network connection Jul 11 22:03:55.825228546 (444h52m55.708642158s) [noise=true , address=104.36.82.14:4222 , piece_id=BUC5EAICHZNODEW5MNAM26KEMQBBVVGIH2P5W7WQSEVNFSB4ZS2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48272->104.36.82.14:4222: use of closed network connection Jul 11 22:03:55.825237626 (444h52m55.708651277s) [noise=true , address=104.36.82.10:4078 , piece_id=QGQ4NH5ZMYFPDIN4MSDEF4SIUZBZ3JCOFKLZ7WWLZEKVZK3QCENQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60164->104.36.82.10:4078: use of closed network connection Jul 11 22:03:55.825297732 (444h52m55.708711344s) [noise=true , address=104.36.82.16:4293 , piece_id=VCINGGXB7VIPIQTGNM5II2EISTTY32OACYGGQZZ7VKKZZZHXBPTA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36322->104.36.82.16:4293: use of closed network connection Jul 11 22:03:55.825363916 (444h52m55.708777529s) [noise=true , address=104.36.82.9:4005 , piece_id=5UDZ7TZ53Q455F7BNK6DLI3A5ONUFOWLABJPV5EB2B72CP3FJL2A, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43396->104.36.82.9:4005: use of closed network connection Jul 11 22:03:55.825421328 (444h52m55.708834939s) [noise=true , address=104.36.82.7:4912 , piece_id=3QWQCGAD4ISSIHW5TQG7QXSNHW3M7J6XBQQSRH3DPTBAZOKREGPA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40356->104.36.82.7:4912: use of closed network connection Jul 11 22:03:55.825463724 (444h52m55.708877335s) [noise=true , address=104.36.82.12:4151 , piece_id=B3CTCC43VMTYQQHMSKFAGTT3EYMXB244IHKDNEE6F4VDJ2XBGBXA, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38246->104.36.82.12:4151: use of closed network connection Jul 11 22:03:55.825477966 (444h52m55.708891585s) [noise=true , address=104.36.82.12:4149 , piece_id=UNAE4IGDADDPLGWRJW6S3JC5AFNSH44REOGKBTV3F7G65YY7EQDQ, seg_pos={0 0}, upload=355755]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54638->104.36.82.12:4149: use of closed network connection Jul 11 22:03:55.825533457 (444h52m55.708947087s) [seg_pos={0 0}, upload=355755]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 11 22:03:55.935680781 (444h52m55.819094399s) [upload=355755]: Done uploading segments... Jul 11 22:03:55.935707519 (444h52m55.819121146s) [upload=355755]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.112497796s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 4.070478821s FAIL (2025-07-11 22:03:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.070506213s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 8.489927325s FAIL (2025-07-11 22:03:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 737s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.488709574 (444h51m30.372123209s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489246417 (444h51m30.372660054s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489303194 (444h51m30.372716825s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489648268 (444h51m30.373061904s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489707285 (444h51m30.373120911s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490095044 (444h51m30.373508684s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490197063 (444h51m30.373610693s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490604300 (444h51m30.374017929s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491042169 (444h51m30.374455796s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491492265 (444h51m30.374905902s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491866085 (444h51m30.375279717s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492126748 (444h51m30.375540385s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492565072 (444h51m30.375978713s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493333026 (444h51m30.376746662s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493702567 (444h51m30.377116204s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494079587 (444h51m30.377493219s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494555851 (444h51m30.377969484s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:31.011808775 (444h51m30.895222393s) [upload=355744]: Next returned nil segment Jul 11 22:02:31.011828319 (444h51m30.895241945s) [upload=355744]: Waiting for error group managing segments... Jul 11 22:02:31.185708662 (444h51m31.069122287s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.203802874 (444h51m31.087216495s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.205879153 (444h51m31.089292766s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.219680112 (444h51m31.103093739s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.222479897 (444h51m31.105893526s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230662544 (444h51m31.114076159s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230778357 (444h51m31.114191976s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.334409626 (444h51m31.217823263s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.339502902 (444h51m31.222916533s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.388877737 (444h51m31.272291372s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389126192 (444h51m31.272539822s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389800397 (444h51m31.273214027s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.426833520 (444h51m31.31024715s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:03:30.023868435 (444h52m29.907282053s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:30.024114203 (444h52m29.90752783s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1pbWgKsDigGSwnWtNHwsJC3srt45YiUhswf2hv9gbn2LSCLkSH, address:104.36.82.18:4374): protocol: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection Jul 11 22:03:30.024186106 (444h52m29.907599732s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:13va7LjDtQV2ob7mrUaJUXc64HmZG3svoDaErmypjYHVAcw1o3, address:104.36.82.40:4962): protocol: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection Jul 11 22:03:30.024207244 (444h52m29.907620871s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:104.36.82.34:4551): protocol: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection Jul 11 22:03:30.024253975 (444h52m29.907667602s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1R66QktZdYxwpsjALURfq9eEvqNG2tfMvBjC5DnackVppZdGpT, address:104.36.82.17:4336): protocol: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection Jul 11 22:03:30.024291452 (444h52m29.907705088s) [seg_pos={0 0}, upload=355744]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:30.137783063 (444h52m30.021196699s) [upload=355744]: Done uploading segments... Jul 11 22:03:30.137819589 (444h52m30.021233226s) [upload=355744]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 8.489952711s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.116611451s FAIL (2025-07-11 22:03:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 737s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.488709574 (444h51m30.372123209s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489246417 (444h51m30.372660054s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489303194 (444h51m30.372716825s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489648268 (444h51m30.373061904s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.489707285 (444h51m30.373120911s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490095044 (444h51m30.373508684s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490197063 (444h51m30.373610693s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.490604300 (444h51m30.374017929s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491042169 (444h51m30.374455796s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491492265 (444h51m30.374905902s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.491866085 (444h51m30.375279717s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492126748 (444h51m30.375540385s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.492565072 (444h51m30.375978713s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493333026 (444h51m30.376746662s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.493702567 (444h51m30.377116204s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494079587 (444h51m30.377493219s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:30.494555851 (444h51m30.377969484s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Uploading piece... Jul 11 22:02:31.011808775 (444h51m30.895222393s) [upload=355744]: Next returned nil segment Jul 11 22:02:31.011828319 (444h51m30.895241945s) [upload=355744]: Waiting for error group managing segments... Jul 11 22:02:31.185708662 (444h51m31.069122287s) [noise=true , address=77.90.31.11:4571 , piece_id=LFRXPKHCSS65V6AE6P3PFDBEZ5DIKY3RFGR4GF5UWPV6P6BHPCZQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.203802874 (444h51m31.087216495s) [noise=true , address=77.90.31.12:4617 , piece_id=WZWXQJGB4MTW4JN44NY5NQCXOWZQSGBX6IIBF56TQKB3UFQ7JEJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.205879153 (444h51m31.089292766s) [noise=true , address=77.90.31.16:4755 , piece_id=KI32C4HB2RCC4EDMM2B6UW6OUPSMAKU5GOA5ANIXTJQD3WLQLXBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.219680112 (444h51m31.103093739s) [noise=true , address=45.91.116.4:4966 , piece_id=K7JXTKTY4UJSCMJUOSWFWHIIZBNCXSP3N5K3INQDF77HDAKHRGIQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.222479897 (444h51m31.105893526s) [noise=true , address=77.90.31.14:4692 , piece_id=WQQ7SNSTHOP4TSKRDDM76VYWJELUWOCH6GSE6XSUE4NVAIMHMPHQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230662544 (444h51m31.114076159s) [noise=true , address=77.90.31.21:4895 , piece_id=IWTSGOW7UNMAJWC3QO3OZWFBUJNWOBTXSCQ6MMMTNUR3BNBSKTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.230778357 (444h51m31.114191976s) [noise=true , address=77.90.31.21:4884 , piece_id=IPGAG6G43WNCIM4E5RIHACCS4R35Q2ZZV4POIEBFG4I6D37IM72A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.334409626 (444h51m31.217823263s) [noise=true , address=79.127.220.101:4066 , piece_id=FWSZRSF3DP2VTJQJKLGKGGZCUP62YXZEWPB5IMIKDU4QHRY46ZJA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.339502902 (444h51m31.222916533s) [noise=true , address=77.90.31.17:4788 , piece_id=TIYZBFZ3JTRNBOFAIWEJWYBMDAOJZYPQ3MWWNUYBXPHQ56VY73WQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.388877737 (444h51m31.272291372s) [noise=true , address=45.91.95.2:4016 , piece_id=OB4WV3YWOQNVUQIUAKFKDS7NF7FX54UEPBZVNNEUAORZUFE2FTXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389126192 (444h51m31.272539822s) [noise=true , address=45.91.95.5:4002 , piece_id=3IZ3MAVFGNZO2C7LPTOY2FTIC6R2DC4OJY6XBJVWMPRE63B4DUEQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.389800397 (444h51m31.273214027s) [noise=true , address=79.127.220.101:4057 , piece_id=QXEVOIUOJRDWTL7FVGAR22HIN4ZKSORAK2BARAQIRCS33BXX5X6Q, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:02:31.426833520 (444h51m31.31024715s) [noise=true , address=45.91.95.4:4019 , piece_id=JBBJKCAXRB2GUXBL6ZJOS3JMJHR3WDDEDQ2LB36BMHO7SPZ6HKBQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: <nil> Jul 11 22:03:30.023868435 (444h52m29.907282053s) [noise=true , address=104.36.82.50:4031 , piece_id=ZTD6S46WOAD2BKVVJ3RCHX55RGJNITAVPZ3ENGQBAQB3OPLW6C2A, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12Ypc3zkMzWs3wqx9QaExy7sHvnSatwYH8Y9rHi7qt3xRELfsAa, address:104.36.82.50:4031): protocol: expected piece hash; context deadline exceeded; EOF Jul 11 22:03:30.024114203 (444h52m29.90752783s) [noise=true , address=104.36.82.18:4374 , piece_id=VJ5O3X2U36FH5W6N2FO7ORNLXW3KNF5QT7H355WQZL2I632HUEAA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1pbWgKsDigGSwnWtNHwsJC3srt45YiUhswf2hv9gbn2LSCLkSH, address:104.36.82.18:4374): protocol: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58430->104.36.82.18:4374: use of closed network connection Jul 11 22:03:30.024186106 (444h52m29.907599732s) [noise=true , address=104.36.82.40:4962 , piece_id=JHVGP44L2LBPL234MI6VSJZYZ6G6C6VIOYGL6ZDIQSFIIX6NJ2UQ, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:13va7LjDtQV2ob7mrUaJUXc64HmZG3svoDaErmypjYHVAcw1o3, address:104.36.82.40:4962): protocol: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54424->104.36.82.40:4962: use of closed network connection Jul 11 22:03:30.024207244 (444h52m29.907620871s) [noise=true , address=104.36.82.34:4551 , piece_id=455MQQEMSNHTSZNEWU37DAFK4JNKKZDDWSUXFMTBOCQJ443PWWXA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:12vR5WRMGoYyzYDZzwELFw6kLG6vtKsUeFM9kUhgZDLGKB6vcXs, address:104.36.82.34:4551): protocol: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54154->104.36.82.34:4551: use of closed network connection Jul 11 22:03:30.024253975 (444h52m29.907667602s) [noise=true , address=104.36.82.17:4336 , piece_id=KCHNTU5TWK4X63FTNR5JTDIFGQOJXDHHCVAVU76YMS4VIAFWRUWA, seg_pos={0 0}, upload=355744]: Done uploading piece. err: ecclient: upload failed (node:1R66QktZdYxwpsjALURfq9eEvqNG2tfMvBjC5DnackVppZdGpT, address:104.36.82.17:4336): protocol: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57364->104.36.82.17:4336: use of closed network connection Jul 11 22:03:30.024291452 (444h52m29.907705088s) [seg_pos={0 0}, upload=355744]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 11 22:03:30.137783063 (444h52m30.021196699s) [upload=355744]: Done uploading segments... Jul 11 22:03:30.137819589 (444h52m30.021233226s) [upload=355744]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.116638344s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXX......................................................XXX] duration: 30.001354859s FAIL (2025-07-11 22:03:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001380521s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.116148261s FAIL (2025-07-11 22:03:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== os={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.051121886 (444h51m45.934535516s) [noise=true , address=45.91.116.4:4937 , piece_id=BV4AKGE6MNEYEVPWZ3DHZUDIKF3BEUORWO26U4D5JNCZOYKGITVQ, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.051390347 (444h51m45.934803973s) [noise=true , address=104.36.82.7:4911 , piece_id=FRFFZP3FI3GHMZF622KIVIB3L3UD2BN6R5VP4GQEVDH4N7OWBZBQ, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.051687123 (444h51m45.935100751s) [noise=true , address=104.36.82.11:4113 , piece_id=GIC2ZMXOOQQLQKME3LQJFW3L6N3ISUC2ZIWYWRWNOHCUEE2RPK2A, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.051970395 (444h51m45.935384024s) [noise=true , address=45.91.116.12:4225 , piece_id=6JO7IBJDPWA3RVDBIRXC7CFPS3BUUYSF4LRGVO6VSCC5G3UBTVAQ, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.055489087 (444h51m45.938902713s) [noise=true , address=104.36.82.2:4732 , piece_id=QZY6COLSB3UK5NK44HGR47EI5FTODG2H2IKHP3V2ATY62E34SBKQ, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.055805940 (444h51m45.939219568s) [noise=true , address=77.90.31.19:4842 , piece_id=GK5YHR7KJ5US2JKUF34W6LJRNX4KAYGCMDYGUBLU4UERCXEXPV5Q, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.056105086 (444h51m45.939518716s) [noise=true , address=104.36.82.10:4078 , piece_id=JC4JF7OCE6BGPTBZ63SUGMPELKOWGFFZLYTLE5G7HMINKGA3LJ6Q, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.056400969 (444h51m45.939814598s) [noise=true , address=77.90.31.18:4806 , piece_id=U2GM4CB247NA6MCXQG4YJMTFGPWU76MY4JFPQWVRFAOYBJIJODQQ, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.056790323 (444h51m45.940203953s) [noise=true , address=104.36.82.14:4221 , piece_id=4VBDIOFZJZALH6LF4M6OGKZ6PKGXSLTPOEOWSMO7SPHAGSMNAYVA, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.057228841 (444h51m45.940642472s) [noise=true , address=104.36.82.12:4150 , piece_id=TZ2DO6ENZMZEHF35Z37LHNCNAZQH4NYUTU2XQHVMUEE55Z74WFAA, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:46.057899840 (444h51m45.941313477s) [noise=true , address=45.91.116.11:4119 , piece_id=D4G2YBS5IBW7N4WPRL35IOD4UBTKR7NZ2YSLUYQE57ENRYXMKD4A, seg_pos={0 0}, upload=355746]: Uploading piece... Jul 11 22:02:47.288521418 (444h51m47.171935036s) [upload=355746]: Next returned nil segment Jul 11 22:02:47.288963404 (444h51m47.172377036s) [upload=355746]: Waiting for error group managing segments... Jul 11 22:02:47.594143047 (444h51m47.477556675s) [noise=true , address=45.91.116.11:4119 , piece_id=D4G2YBS5IBW7N4WPRL35IOD4UBTKR7NZ2YSLUYQE57ENRYXMKD4A, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:02:47.736017804 (444h51m47.619431424s) [noise=true , address=77.90.31.18:4806 , piece_id=U2GM4CB247NA6MCXQG4YJMTFGPWU76MY4JFPQWVRFAOYBJIJODQQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:02:47.981604187 (444h51m47.865017806s) [noise=true , address=77.90.31.13:4628 , piece_id=JFRPDEOH63O7OX6BSECWH4CW6GUEJTX5PAZ42TJVF52FOKJ2Z5IQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:02:48.108640018 (444h51m47.992053636s) [noise=true , address=77.90.31.18:4805 , piece_id=IYK2MR4AAIR4BASENLDQKCNCXRATBIBFYZZHV7TNUORZODTN2LRA, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:02:49.241941915 (444h51m49.125355532s) [noise=true , address=77.90.31.19:4842 , piece_id=GK5YHR7KJ5US2JKUF34W6LJRNX4KAYGCMDYGUBLU4UERCXEXPV5Q, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:02:49.601843612 (444h51m49.485257236s) [noise=true , address=45.91.116.4:4937 , piece_id=BV4AKGE6MNEYEVPWZ3DHZUDIKF3BEUORWO26U4D5JNCZOYKGITVQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:03:10.976460075 (444h52m10.859873694s) [noise=true , address=45.91.116.12:4225 , piece_id=6JO7IBJDPWA3RVDBIRXC7CFPS3BUUYSF4LRGVO6VSCC5G3UBTVAQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: <nil> Jul 11 22:03:15.661769458 (444h52m15.545183094s) [noise=true , address=104.36.82.2:4732 , piece_id=QZY6COLSB3UK5NK44HGR47EI5FTODG2H2IKHP3V2ATY62E34SBKQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: write tcp 139.59.144.22:43072->104.36.82.2:4732: use of closed network connection; write tcp 139.59.144.22:43072->104.36.82.2:4732: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43072->104.36.82.2:4732: use of closed network connection Jul 11 22:03:15.662013531 (444h52m15.545427157s) [noise=true , address=104.36.82.16:4294 , piece_id=7RLKZFQLKFFEAVXXSSPENNFCMNG2N5YHL7ZCOV66YIQUZSBVS44Q, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 139.59.144.22:56800->104.36.82.16:4294: use of closed network connection; write tcp 139.59.144.22:56800->104.36.82.16:4294: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56800->104.36.82.16:4294: use of closed network connection Jul 11 22:03:15.662051158 (444h52m15.545464771s) [noise=true , address=104.36.82.14:4221 , piece_id=4VBDIOFZJZALH6LF4M6OGKZ6PKGXSLTPOEOWSMO7SPHAGSMNAYVA, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 139.59.144.22:51974->104.36.82.14:4221: use of closed network connection; write tcp 139.59.144.22:51974->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51974->104.36.82.14:4221: use of closed network connection Jul 11 22:03:15.662254955 (444h52m15.54566858s) [noise=true , address=104.36.82.7:4911 , piece_id=FRFFZP3FI3GHMZF622KIVIB3L3UD2BN6R5VP4GQEVDH4N7OWBZBQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 139.59.144.22:38326->104.36.82.7:4911: use of closed network connection; write tcp 139.59.144.22:38326->104.36.82.7:4911: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38326->104.36.82.7:4911: use of closed network connection Jul 11 22:03:15.662338582 (444h52m15.545752195s) [noise=true , address=104.36.82.10:4078 , piece_id=JC4JF7OCE6BGPTBZ63SUGMPELKOWGFFZLYTLE5G7HMINKGA3LJ6Q, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:44998->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:44998->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44998->104.36.82.10:4078: use of closed network connection Jul 11 22:03:15.662406843 (444h52m15.545820455s) [noise=true , address=104.36.82.11:4114 , piece_id=PSISMBWXRXELPF6TZ6WW4ZFXWHHXQEN3OQVCBBXKY4XZ2GEH2NWQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: write tcp 139.59.144.22:58552->104.36.82.11:4114: use of closed network connection; write tcp 139.59.144.22:58552->104.36.82.11:4114: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58552->104.36.82.11:4114: use of closed network connection Jul 11 22:03:15.662411026 (444h52m15.545824653s) [noise=true , address=104.36.82.11:4113 , piece_id=GIC2ZMXOOQQLQKME3LQJFW3L6N3ISUC2ZIWYWRWNOHCUEE2RPK2A, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 139.59.144.22:50720->104.36.82.11:4113: use of closed network connection; write tcp 139.59.144.22:50720->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50720->104.36.82.11:4113: use of closed network connection Jul 11 22:03:15.662459109 (444h52m15.545872722s) [noise=true , address=104.36.82.12:4150 , piece_id=TZ2DO6ENZMZEHF35Z37LHNCNAZQH4NYUTU2XQHVMUEE55Z74WFAA, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:37274->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:37274->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37274->104.36.82.12:4150: use of closed network connection Jul 11 22:03:15.662496172 (444h52m15.545909798s) [noise=true , address=104.36.82.10:4079 , piece_id=D7AAZSMNT52AZR2T5ITTS6KLN7UQ4ZJGYWAHHTRF37UTF5KE4MTQ, seg_pos={0 0}, upload=355746]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: write tcp 139.59.144.22:51976->104.36.82.10:4079: use of closed network connection; write tcp 139.59.144.22:51976->104.36.82.10:4079: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51976->104.36.82.10:4079: use of closed network connection Jul 11 22:03:15.662527258 (444h52m15.545940887s) [seg_pos={0 0}, upload=355746]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 11 22:03:15.776359568 (444h52m15.659773186s) [upload=355746]: Done uploading segments... Jul 11 22:03:15.776391617 (444h52m15.659805246s) [upload=355746]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.11625265s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 8.91718167s FAIL (2025-07-11 22:03:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.917238339s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.XXX......................................................XX] duration: 30.000833054s FAIL (2025-07-11 22:02:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000857607s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.005214126s FAIL (2025-07-11 22:02:29 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload=355730]: Got next segment. Inline: false Jul 11 22:01:30.131248876 (444h50m30.014662503s) [upload=355730]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:01:30.131256577 (444h50m30.01467021s) [upload=355730]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:01:30.328232791 (444h50m30.211646489s) [seg_pos={0 0}, upload=355730]: Begin upload segment... Jul 11 22:01:30.328304778 (444h50m30.211718397s) [seg_pos={0 0}, upload=355730]: Done begin upload segment. Jul 11 22:01:30.328356063 (444h50m30.211769697s) [noise=true , address=45.91.116.7:4015 , piece_id=Q7MCQCOVOBO5QTJPBVYIE5KQNDCVT7MPGHHDVKH2NNY2IIYGSZRA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.328925827 (444h50m30.212339461s) [noise=true , address=79.127.220.101:4072 , piece_id=5W3Z3BEGCJCZ2AE7QXZR7NCSFXPFBMCIAUVSGRYIPKS5WZ342CLA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.329535879 (444h50m30.212949506s) [noise=true , address=64.49.13.27:4963 , piece_id=XBYXGIUW5PCKAKF24CZZZGPLALS3HYUB3LDMF3QPVD246JCWREGQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.329960002 (444h50m30.213373636s) [noise=true , address=104.36.82.14:4228 , piece_id=6FSDK5IK6LW3KVTK5E3X5ML3VTDML423NCTTH5GCGLWGQZ23ZGTQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.330074642 (444h50m30.213488307s) [noise=true , address=104.36.82.13:4203 , piece_id=L5AFCHD2JUM7AF6WWQZTH5SE7IQWZJWWXZ62XJV45DCONBCOZO2A, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.330460305 (444h50m30.213873948s) [noise=true , address=104.36.82.7:4913 , piece_id=3ZVJO474BWWADKXNYOQXIMTO25Y4DW7ZD5WGK7FKJNF2LOKUAI7Q, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.330560444 (444h50m30.213974082s) [noise=true , address=45.91.116.9:4290 , piece_id=6BIMHDRXU56NMAQGGRBQFDK6CVJ2F3WDJX4A24WTW3RFSOOASULA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.330944051 (444h50m30.214357683s) [noise=true , address=77.90.31.17:4782 , piece_id=BCK4LB77VKBM5PNYQMQ47RAPJ3OFTPIH5WJ6CELBHG5NADUBHQUQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.330953702 (444h50m30.214367341s) [noise=true , address=77.90.31.16:4737 , piece_id=E2MY3ECGO6FUH5YIXX7VDVEWFP3P37MMUXESWE3Z3JMIE2TDIGSQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331029128 (444h50m30.214442766s) [noise=true , address=45.91.116.7:4039 , piece_id=3OQXULCAHZB2VNM55NZY3ICGTFZYRNGKNZWKZEICNIMZJYBQWBVA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331378972 (444h50m30.214792607s) [noise=true , address=77.90.31.21:4897 , piece_id=BA5PAGGLE6YYIOJRNVRMBE7BHX3TJJOBDYNDDP47EFGBXHPJVGBQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331428352 (444h50m30.214841988s) [noise=true , address=45.91.116.4:4966 , piece_id=VIYAVCV2UG5BUGDPOYUAXA4XTOV5WHCT2UETCM33OJ6UEOP2EBRA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331720882 (444h50m30.215134517s) [noise=true , address=104.36.82.40:4965 , piece_id=JXYGIESVPUYKF3WXCXM7XNZ3STLYD7OG2MWGUV5FXIBR5ZZWCJ3A, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331867483 (444h50m30.215281123s) [noise=true , address=45.91.95.2:4015 , piece_id=WEFNO4HJIPDW3FUFTZC5ZLXE6ZMU3BZCQWLCM66GMOBHAU5ANCWQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.331994001 (444h50m30.215407632s) [noise=true , address=45.91.95.3:4027 , piece_id=OPBFOV4J46APD5BIY4Q2A3JN2XKZV5AHCELMQKV2KNLP4DX2WY6Q, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.332270773 (444h50m30.21568441s) [noise=true , address=77.90.31.15:4701 , piece_id=BEZN46GSIBIVXYVAIIP7U5HRQYHP6U7KADW5UTYL27GLV6LBJ3QA, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.332686499 (444h50m30.216100135s) [noise=true , address=45.91.95.5:4009 , piece_id=SDAEO4EIX66IEWQN42O2VJUR55PF7U3RHWGKEN4CATAKIZA2UU5A, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.333240455 (444h50m30.21665409s) [noise=true , address=104.36.82.41:4783 , piece_id=2SESWGHUDQRBDLWD5M2G7EWLMS2NX3FP4PORZSGOEWTZGZQC7ILQ, seg_pos={0 0}, upload=355730]: Uploading piece... Jul 11 22:01:30.940907943 (444h50m30.824321583s) [upload=355730]: Next returned nil segment Jul 11 22:01:30.940932405 (444h50m30.824346044s) [upload=355730]: Waiting for error group managing segments... Jul 11 22:01:31.089793596 (444h50m30.973207215s) [noise=true , address=104.36.82.41:4783 , piece_id=2SESWGHUDQRBDLWD5M2G7EWLMS2NX3FP4PORZSGOEWTZGZQC7ILQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.137803130 (444h50m31.02121675s) [noise=true , address=104.36.82.14:4228 , piece_id=6FSDK5IK6LW3KVTK5E3X5ML3VTDML423NCTTH5GCGLWGQZ23ZGTQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.189597866 (444h50m31.073011478s) [noise=true , address=104.36.82.40:4965 , piece_id=JXYGIESVPUYKF3WXCXM7XNZ3STLYD7OG2MWGUV5FXIBR5ZZWCJ3A, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.221033801 (444h50m31.104447421s) [noise=true , address=79.127.220.101:4072 , piece_id=5W3Z3BEGCJCZ2AE7QXZR7NCSFXPFBMCIAUVSGRYIPKS5WZ342CLA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.681991641 (444h50m31.56540526s) [noise=true , address=64.49.13.27:4963 , piece_id=XBYXGIUW5PCKAKF24CZZZGPLALS3HYUB3LDMF3QPVD246JCWREGQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.761237360 (444h50m31.644650987s) [noise=true , address=45.91.116.9:4290 , piece_id=6BIMHDRXU56NMAQGGRBQFDK6CVJ2F3WDJX4A24WTW3RFSOOASULA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:31.949637778 (444h50m31.833051397s) [noise=true , address=45.91.116.7:4039 , piece_id=3OQXULCAHZB2VNM55NZY3ICGTFZYRNGKNZWKZEICNIMZJYBQWBVA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:32.459024187 (444h50m32.342437811s) [noise=true , address=104.36.82.7:4913 , piece_id=3ZVJO474BWWADKXNYOQXIMTO25Y4DW7ZD5WGK7FKJNF2LOKUAI7Q, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:32.816162876 (444h50m32.699576503s) [noise=true , address=104.36.82.13:4203 , piece_id=L5AFCHD2JUM7AF6WWQZTH5SE7IQWZJWWXZ62XJV45DCONBCOZO2A, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:32.816622627 (444h50m32.700036247s) [noise=true , address=45.91.116.7:4015 , piece_id=Q7MCQCOVOBO5QTJPBVYIE5KQNDCVT7MPGHHDVKH2NNY2IIYGSZRA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:33.045081934 (444h50m32.928495549s) [noise=true , address=77.90.31.21:4897 , piece_id=BA5PAGGLE6YYIOJRNVRMBE7BHX3TJJOBDYNDDP47EFGBXHPJVGBQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:33.607467890 (444h50m33.490881508s) [noise=true , address=77.90.31.15:4701 , piece_id=BEZN46GSIBIVXYVAIIP7U5HRQYHP6U7KADW5UTYL27GLV6LBJ3QA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:35.255172697 (444h50m35.138586311s) [noise=true , address=77.90.31.17:4782 , piece_id=BCK4LB77VKBM5PNYQMQ47RAPJ3OFTPIH5WJ6CELBHG5NADUBHQUQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:35.643248755 (444h50m35.526662382s) [noise=true , address=45.91.95.3:4027 , piece_id=OPBFOV4J46APD5BIY4Q2A3JN2XKZV5AHCELMQKV2KNLP4DX2WY6Q, seg_pos={0 0}, upload=355730]: Done uploading piece. err: <nil> Jul 11 22:01:35.643306637 (444h50m35.526720264s) [seg_pos={0 0}, upload=355730]: Segment reached optimal threshold of 14 pieces. Jul 11 22:01:35.643504132 (444h50m35.526917763s) [noise=true , address=45.91.95.5:4009 , piece_id=SDAEO4EIX66IEWQN42O2VJUR55PF7U3RHWGKEN4CATAKIZA2UU5A, seg_pos={0 0}, upload=355730]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN): protocol: expected piece hash; context canceled; EOF Jul 11 22:01:35.643769929 (444h50m35.527183558s) [noise=true , address=45.91.95.2:4015 , piece_id=WEFNO4HJIPDW3FUFTZC5ZLXE6ZMU3BZCQWLCM66GMOBHAU5ANCWQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1swt4fnsU2PmLbnozfXxbQPNbdDVjgdfpxFFR4VsvAdKcf5ZUj): protocol: write tcp 139.59.144.22:59746->45.91.95.2:4015: use of closed network connection; write tcp 139.59.144.22:59746->45.91.95.2:4015: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59746->45.91.95.2:4015: use of closed network connection Jul 11 22:01:35.643854515 (444h50m35.52726815s) [noise=true , address=77.90.31.16:4737 , piece_id=E2MY3ECGO6FUH5YIXX7VDVEWFP3P37MMUXESWE3Z3JMIE2TDIGSQ, seg_pos={0 0}, upload=355730]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM): protocol: expected piece hash; context canceled; EOF Jul 11 22:01:35.643967977 (444h50m35.527381606s) [noise=true , address=45.91.116.4:4966 , piece_id=VIYAVCV2UG5BUGDPOYUAXA4XTOV5WHCT2UETCM33OJ6UEOP2EBRA, seg_pos={0 0}, upload=355730]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DSHTtpwcx1h1utGXNou11K7Tcg1ZhkkMdMgY6BDD2ifG6LQ7S): protocol: expected piece hash; context canceled; EOF Jul 11 22:01:35.643995156 (444h50m35.52740879s) [seg_pos={0 0}, upload=355730]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 11 22:01:35.644095143 (444h50m35.527508779s) [upload=355730]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 11 22:01:35.644101401 (444h50m35.527515035s) [upload=355730]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 11 22:01:35.799217788 (444h50m35.682631415s) [upload=355730]: Done uploading segments... Jul 11 22:01:35.799242206 (444h50m35.682655827s) [upload=355730]: Upload finished. err: <nil> Duration: 1m0.005247926s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXX......................................................X] duration: 30.000468822s FAIL (2025-07-11 22:02:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000489798s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.115009932s FAIL (2025-07-11 22:02:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 50m35.033301322s) [upload=355732]: Got next segment. Inline: false Jul 11 22:01:35.151129627 (444h50m35.034543264s) [upload=355732]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 11 22:01:35.151145534 (444h50m35.03455917s) [upload=355732]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 11 22:01:35.263865368 (444h50m35.147278985s) [seg_pos={0 0}, upload=355732]: Begin upload segment... Jul 11 22:01:35.263930002 (444h50m35.147343617s) [seg_pos={0 0}, upload=355732]: Done begin upload segment. Jul 11 22:01:35.263958148 (444h50m35.147371765s) [noise=true , address=104.36.82.12:4149 , piece_id=XN54HBVSSEM7E63XJ6NMNSF7I2Q7OCKC3TU6H4KTAOOKPCZSZO7A, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.264493568 (444h50m35.147907197s) [noise=true , address=77.90.31.21:4877 , piece_id=YIRE4BFZHRBIOOBV6XJRNMDB55SYF2H35HL7NFJNTKSOQAYUUN2Q, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.264828919 (444h50m35.148242536s) [noise=true , address=104.36.82.5:4839 , piece_id=XPOLXQLQLC2VE3DKXRVQ3TKIE7MM3OSEQSXTZHIIOREZOJISG2PA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.265397295 (444h50m35.148810911s) [noise=true , address=77.90.31.10:4517 , piece_id=3R5UZPYWQNTJOO6KZQR2XQXQMAVA4TSMTDOSL2XI32IA53P5NKXA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.265803320 (444h50m35.149216947s) [noise=true , address=77.90.31.10:4519 , piece_id=4KD6UHNNZDKYGFAJNR7IY7QX7SILJRFWWV4TQMEK5PWHNFNTXNJQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.266115373 (444h50m35.14952901s) [noise=true , address=45.91.116.13:4155 , piece_id=I3BC5ORAPAA22GURH2UIEJPV3YF3277HREBRAS2VBG7OV2UGNIFQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.266700154 (444h50m35.150113791s) [noise=true , address=104.36.82.2:4729 , piece_id=P4JAQHT6XTHZWVCXDP5XJNPAFLJJWOPOIY4OYOFQHW4VKPJUA3BQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.267135986 (444h50m35.150549622s) [noise=true , address=45.91.116.6:4973 , piece_id=K2FQU3YYS7UPR6FCYVIBM4K74XF2X5EXPQS4WGQ3SSXIHZWD5QJQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.267593165 (444h50m35.151006801s) [noise=true , address=104.36.82.15:4257 , piece_id=UPXB3X7NR62ZVSBVS2D6ID6RXQ4XLTWAF2S2R3FCLQGDW5M7QXOA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.268004236 (444h50m35.151417872s) [noise=true , address=104.36.82.16:4295 , piece_id=IW6NQNNOW73HFPSAAYDKFCBIGXTSLRMVLI7TJXOOMPXHLJ7AICRQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.268043186 (444h50m35.151456819s) [noise=true , address=104.36.82.14:4223 , piece_id=LGIQGPBTCMP65C7BOBCPWEHPQ6HPSNYURC457S4CTHAGP7XHENLA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.268444295 (444h50m35.151857922s) [noise=true , address=104.36.82.9:4006 , piece_id=QHPJDFXRP5YE4ZCLIG37I4QG4TMGUHDYBSYKJ6ELC322HYBV4XNQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.268753353 (444h50m35.15216698s) [noise=true , address=104.36.82.9:4007 , piece_id=VMHEK63OJSBPPJPP7CWF7K3IP2FLC2IXRUW4X3PZWLTDCYZA42SQ, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.269219456 (444h50m35.152633092s) [noise=true , address=104.36.82.8:4969 , piece_id=NUMPH7XK7FVMXTAGEJMMWRSOA2S6QOTEAQ2BC5GYTUX3O2L6PD4Q, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.269660983 (444h50m35.153074619s) [noise=true , address=104.36.82.11:4114 , piece_id=KPOOZ4BQT754XBWGIL4RPAZFVJ2P3LU2CSKP33YGCIOS3KW7YKWA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.270215644 (444h50m35.153629282s) [noise=true , address=104.36.82.6:4876 , piece_id=YKIHVI6UAGJPQN6VR22K5E37U6S6BGBQF6HXETD7GDND4L426KJA, seg_pos={0 0}, upload=355732]: Uploading piece... Jul 11 22:01:35.905294754 (444h50m35.78870839s) [upload=355732]: Next returned nil segment Jul 11 22:01:35.905317897 (444h50m35.788731534s) [upload=355732]: Waiting for error group managing segments... Jul 11 22:01:36.037954645 (444h50m35.921368265s) [noise=true , address=104.36.82.9:4006 , piece_id=QHPJDFXRP5YE4ZCLIG37I4QG4TMGUHDYBSYKJ6ELC322HYBV4XNQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:36.045843631 (444h50m35.929257251s) [noise=true , address=104.36.82.9:4007 , piece_id=VMHEK63OJSBPPJPP7CWF7K3IP2FLC2IXRUW4X3PZWLTDCYZA42SQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:36.066539953 (444h50m35.949953582s) [noise=true , address=104.36.82.2:4729 , piece_id=P4JAQHT6XTHZWVCXDP5XJNPAFLJJWOPOIY4OYOFQHW4VKPJUA3BQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:36.086743355 (444h50m35.970156973s) [noise=true , address=104.36.82.16:4295 , piece_id=IW6NQNNOW73HFPSAAYDKFCBIGXTSLRMVLI7TJXOOMPXHLJ7AICRQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:36.093957440 (444h50m35.977371059s) [noise=true , address=104.36.82.8:4969 , piece_id=NUMPH7XK7FVMXTAGEJMMWRSOA2S6QOTEAQ2BC5GYTUX3O2L6PD4Q, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:36.963571378 (444h50m36.846984994s) [noise=true , address=77.90.31.10:4519 , piece_id=4KD6UHNNZDKYGFAJNR7IY7QX7SILJRFWWV4TQMEK5PWHNFNTXNJQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:37.074473871 (444h50m36.957887492s) [noise=true , address=77.90.31.10:4517 , piece_id=3R5UZPYWQNTJOO6KZQR2XQXQMAVA4TSMTDOSL2XI32IA53P5NKXA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:39.081438211 (444h50m38.96485184s) [noise=true , address=77.90.31.21:4877 , piece_id=YIRE4BFZHRBIOOBV6XJRNMDB55SYF2H35HL7NFJNTKSOQAYUUN2Q, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:41.711617388 (444h50m41.595031009s) [noise=true , address=45.91.116.6:4973 , piece_id=K2FQU3YYS7UPR6FCYVIBM4K74XF2X5EXPQS4WGQ3SSXIHZWD5QJQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:01:42.842674861 (444h50m42.726088481s) [noise=true , address=45.91.116.13:4155 , piece_id=I3BC5ORAPAA22GURH2UIEJPV3YF3277HREBRAS2VBG7OV2UGNIFQ, seg_pos={0 0}, upload=355732]: Done uploading piece. err: <nil> Jul 11 22:02:04.878453138 (444h51m4.761866756s) [noise=true , address=104.36.82.5:4839 , piece_id=XPOLXQLQLC2VE3DKXRVQ3TKIE7MM3OSEQSXTZHIIOREZOJISG2PA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: write tcp 139.59.144.22:46584->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:46584->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46584->104.36.82.5:4839: use of closed network connection Jul 11 22:02:04.878535503 (444h51m4.761949122s) [noise=true , address=104.36.82.12:4149 , piece_id=XN54HBVSSEM7E63XJ6NMNSF7I2Q7OCKC3TU6H4KTAOOKPCZSZO7A, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: write tcp 139.59.144.22:60250->104.36.82.12:4149: use of closed network connection; write tcp 139.59.144.22:60250->104.36.82.12:4149: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60250->104.36.82.12:4149: use of closed network connection Jul 11 22:02:04.878620605 (444h51m4.762034221s) [noise=true , address=104.36.82.15:4257 , piece_id=UPXB3X7NR62ZVSBVS2D6ID6RXQ4XLTWAF2S2R3FCLQGDW5M7QXOA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:44414->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:44414->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44414->104.36.82.15:4257: use of closed network connection Jul 11 22:02:04.878882780 (444h51m4.76229641s) [noise=true , address=104.36.82.14:4223 , piece_id=LGIQGPBTCMP65C7BOBCPWEHPQ6HPSNYURC457S4CTHAGP7XHENLA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:39886->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:39886->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39886->104.36.82.14:4223: use of closed network connection Jul 11 22:02:04.879169955 (444h51m4.762583583s) [noise=true , address=104.36.82.11:4114 , piece_id=KPOOZ4BQT754XBWGIL4RPAZFVJ2P3LU2CSKP33YGCIOS3KW7YKWA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:104.36.82.11:4114): protocol: read tcp 139.59.144.22:56970->104.36.82.11:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 11 22:02:04.879538306 (444h51m4.762951934s) [noise=true , address=104.36.82.6:4876 , piece_id=YKIHVI6UAGJPQN6VR22K5E37U6S6BGBQF6HXETD7GDND4L426KJA, seg_pos={0 0}, upload=355732]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:37596->104.36.82.6:4876: use of closed network connection; context deadline exceeded Jul 11 22:02:04.879568177 (444h51m4.76298181s) [seg_pos={0 0}, upload=355732]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 11 22:02:04.990892131 (444h51m4.87430575s) [upload=355732]: Done uploading segments... Jul 11 22:02:04.990922932 (444h51m4.874336551s) [upload=355732]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.115037752s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.894253868s FAIL (2025-07-11 22:01:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.894324952s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 11.911340354s FAIL (2025-07-11 22:01:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.91138124s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 4.033703528s FAIL (2025-07-11 22:01:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.033739186s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XXX.....................................................] duration: 5.281958118s FAIL (2025-07-11 22:00:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.281985233s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.853768407s FAIL (2025-07-11 22:00:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.853821015s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XXX....................................................] duration: 2.733060102s FAIL (2025-07-11 21:59:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.733092866s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 8.12246218s FAIL (2025-07-11 21:59:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.122490255s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XXX...................................................] duration: 2.623533078s FAIL (2025-07-11 21:58:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.623563616s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.998113682s FAIL (2025-07-11 21:58:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.998142639s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XXX..................................................] duration: 2.771126095s FAIL (2025-07-11 21:58:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.771231105s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 3.06429784s FAIL (2025-07-11 21:57:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.064322117s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XXX.................................................] duration: 2.69434622s FAIL (2025-07-11 21:56:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.694370042s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.715912029s FAIL (2025-07-11 21:56:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.716457263s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XXX................................................] duration: 3.144858934s FAIL (2025-07-11 21:56:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.144887577s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 20.778958094s FAIL (2025-07-11 21:55:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.778979362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XXX...............................................] duration: 2.767328077s FAIL (2025-07-11 21:55:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.767354821s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XXX..............................................] duration: 2.716757826s FAIL (2025-07-11 21:55:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.716784031s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.797288958s FAIL (2025-07-11 21:54:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.797408316s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............XXX.............................................] duration: 2.835333s FAIL (2025-07-11 21:54:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.835375996s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............XXX............................................] duration: 2.586253181s FAIL (2025-07-11 21:53:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.58627647s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.858060168s FAIL (2025-07-11 21:53:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.858083417s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration: 2.65311251s FAIL (2025-07-11 21:53:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.653409215s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration: 2.583277377s FAIL (2025-07-11 21:52:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.583303883s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 3.012341527s FAIL (2025-07-11 21:52:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.012366851s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XXX.........................................] duration: 2.617215752s FAIL (2025-07-11 21:52:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.617241214s edge gateway us-select-1

1
...
3549
3550
3551
3552
3553
...
3729