Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.99523685s FAIL (2025-10-07 00:02:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.995268346s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.812853378s FAIL (2025-10-07 00:02:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.812883792s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.96904619s FAIL (2025-10-07 00:00:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.96907995s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 3.122620451s FAIL (2025-10-06 23:59:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.122660072s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 3.141581287s FAIL (2025-10-06 23:58:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.141611564s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.580262346s FAIL (2025-10-06 23:58:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.580289644s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.811653438s FAIL (2025-10-06 23:58:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.811683289s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X.........X.................................................] duration: 3.002657595s FAIL (2025-10-06 23:58:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.002695628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.843634185s FAIL (2025-10-06 23:57:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.843665799s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.676666614s FAIL (2025-10-06 23:57:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.676701951s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X.........X................................................] duration: 2.789371064s FAIL (2025-10-06 23:57:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.789420447s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X.........X...............................................] duration: 2.777442767s FAIL (2025-10-06 23:56:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.777465468s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.568671564s FAIL (2025-10-06 23:56:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.568701325s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 2.394987315s FAIL (2025-10-06 23:56:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.395015809s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.700333843s FAIL (2025-10-06 23:56:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.700368408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration: 2.371459469s FAIL (2025-10-06 23:56:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.371494383s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X.........X..............................................] duration: 2.849297792s FAIL (2025-10-06 23:56:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.849504403s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.411441946s FAIL (2025-10-06 23:55:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.411472798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..X......................................................] duration: 2.406006519s FAIL (2025-10-06 23:55:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.406047827s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X...X.........X.............................................] duration: 3.010593704s FAIL (2025-10-06 23:55:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.010617294s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 2.527081107s FAIL (2025-10-06 23:55:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.527110573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..X..X.....................................................] duration: 2.936094048s FAIL (2025-10-06 23:55:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.936125761s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.X...X.........X............................................] duration: 2.823258762s FAIL (2025-10-06 23:55:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.823283683s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX...X......................................................] duration: 2.883903815s FAIL (2025-10-06 23:55:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.884019081s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X..X..X....................................................] duration: 2.722869492s FAIL (2025-10-06 23:54:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.722921282s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X..X..X...................................................] duration: 2.544432166s FAIL (2025-10-06 23:54:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.544475228s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..X...X.........X...........................................] duration: 3.409092465s FAIL (2025-10-06 23:54:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.409133186s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..X..X..................................................] duration: 2.894733214s FAIL (2025-10-06 23:53:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.894763654s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX...X.....................................................] duration: 3.024122692s FAIL (2025-10-06 23:53:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 3.024154037s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [X..X...X.........X..........................................] duration: 2.893085733s FAIL (2025-10-06 23:53:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.893149786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX...X....................................................] duration: 2.867830435s FAIL (2025-10-06 23:53:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.867921435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [X...X..X..X.................................................] duration: 2.630592947s FAIL (2025-10-06 23:53:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.630624026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XX...X...................................................] duration: 3.609718357s FAIL (2025-10-06 23:53:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 3.609777619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X...X..X..X................................................] duration: 2.360647521s FAIL (2025-10-06 23:52:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.360678492s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.530454465s FAIL (2025-10-06 23:52:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.530496526s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XX...X..................................................] duration: 2.591399095s FAIL (2025-10-06 23:52:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.59143088s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.X..X...X.........X.........................................] duration: 2.822853539s FAIL (2025-10-06 23:52:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.822902603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X...X..X..X...............................................] duration: 2.468683696s FAIL (2025-10-06 23:52:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.468715057s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..X..X...X.........X........................................] duration: 3.381725379s FAIL (2025-10-06 23:52:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.381755136s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X...X..X..X..............................................] duration: 2.414521949s FAIL (2025-10-06 23:52:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.414554301s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XX...X.................................................] duration: 2.715490024s FAIL (2025-10-06 23:52:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.715527861s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.81086281s FAIL (2025-10-06 23:51:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.810894529s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...X..X...X.........X.......................................] duration: 2.806445211s FAIL (2025-10-06 23:51:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.80647472s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.817949574s FAIL (2025-10-06 23:51:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.817964145s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X...X..X..X.............................................] duration: 2.582655631s FAIL (2025-10-06 23:51:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.582698353s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....X..X...X.........X......................................] duration: 3.280198088s FAIL (2025-10-06 23:51:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.280225369s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XX...X................................................] duration: 2.683112434s FAIL (2025-10-06 23:50:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.683139562s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.942136406s FAIL (2025-10-06 23:50:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.94217332s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X......XX...X...............................................] duration: 2.606417301s FAIL (2025-10-06 23:50:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 21h44m24.814944914s) [upload=283130]: Uploading segments... Oct 6 23:14:37.011693674 (321h44m24.815158266s) [upload=283130]: Got next segment. Inline: false Oct 6 23:14:37.011759368 (321h44m24.815223953s) [upload=283130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:37.011765154 (321h44m24.815229745s) [upload=283130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:37.123040562 (321h44m24.926505157s) [seg_pos={0 0}, upload=283130]: Begin upload segment... Oct 6 23:14:37.123145277 (321h44m24.926609876s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.130217212 (321h44m24.933681813s) [seg_pos={0 0}, upload=283130]: Done begin upload segment. Oct 6 23:14:37.130354898 (321h44m24.9338195s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.131128462 (321h44m24.934593068s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.132696720 (321h44m24.93616132s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133596422 (321h44m24.937061023s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.133713183 (321h44m24.937177794s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134356842 (321h44m24.937821441s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134498352 (321h44m24.937962958s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.134905544 (321h44m24.938370149s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135151391 (321h44m24.938615995s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135465599 (321h44m24.938930197s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.135773195 (321h44m24.939237796s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136003505 (321h44m24.939468108s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.136665066 (321h44m24.940129665s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137195600 (321h44m24.940660202s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.137728259 (321h44m24.94119286s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Uploading piece... Oct 6 23:14:37.785112144 (321h44m25.588576747s) [upload=283130]: Next returned nil segment Oct 6 23:14:37.785138139 (321h44m25.58860274s) [upload=283130]: Waiting for error group managing segments... Oct 6 23:14:37.912422203 (321h44m25.715886795s) [noise=true , address=145.79.227.18:4007 , piece_id=2QZNXC7KM3XJVZEEXSFOAFV2BLKBH57HE3B6KJ6DH2K5TQ7DNJCQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913097624 (321h44m25.716562216s) [noise=true , address=145.79.227.2:4115 , piece_id=FBP7E4QCAAMRKV46VDQAYCYZ3FVGVFL7KSU54X4NNOYBAYM6RZXA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913185687 (321h44m25.716650272s) [noise=true , address=145.79.227.19:4078 , piece_id=FAEFPSXXUIQE3CI5HTFKIVLUECVWYGKMIRZZYFTU4CXXUFDQ3K7Q, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913592252 (321h44m25.71705685s) [noise=true , address=145.79.227.18:4006 , piece_id=2ZH7VYG2Y2HCLIIT6UVS6UDAAWVWT3HGVGNZ2PEQI2JGQ3VJPBMA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.913974583 (321h44m25.717439169s) [noise=true , address=145.79.227.5:4223 , piece_id=MVLUJQ7ONSZIQ7TJQ5WWKKH5663CUMYRPTEXWY7T6TVZQ7ABPJGA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.916344832 (321h44m25.719809414s) [noise=true , address=145.79.227.3:4151 , piece_id=IKJPGOVYDETCQZEKVAQ72OROPK2W3MKNYEYXR2R7WRMAXQQ2QHQQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.921531945 (321h44m25.724996523s) [noise=true , address=145.79.227.15:4874 , piece_id=WCPDPEHBTWAIJIKLEVMN25KVEEDIQAQI75NBTQFSYZKXB4U4TQRQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:14:37.932105629 (321h44m25.735570208s) [noise=true , address=145.79.227.3:4150 , piece_id=HZ6KN5IV56KAOUGNSAKZFBST6A7PZAZC7CLALD2BF6OWUM72BVBA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: <nil> Oct 6 23:15:06.715286602 (321h44m54.518751199s) [noise=true , address=45.91.116.11:4117 , piece_id=AO5N4ME4S3VAJ2SACTO3H47CT52C5FYRBXOOFSAVUX3OQNE7QSSA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715572308 (321h44m54.519036908s) [noise=true , address=45.91.116.4:4940 , piece_id=LAHVXBQ3UYSKIUCFRDVSOCJRR4FRS7ZDTSH5LUJHBTTOOH6X7KKQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715766153 (321h44m54.519230745s) [noise=true , address=45.91.116.4:4937 , piece_id=IY3XJTOOIQKKJX2Q6KLAFQEN3K4BGGZAJLP6TLLC5XWAVCN4EZAA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.715993973 (321h44m54.519458565s) [noise=true , address=45.91.116.9:4262 , piece_id=P5VZRXD7JTE32FHSR2WZPQVDHGAQYLPKYJER43MTAWU5TZFYFIRA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 139.59.144.22:52202->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716157018 (321h44m54.519621613s) [noise=true , address=45.91.116.8:4045 , piece_id=PGZD2XHSBC3QTAKL7HB2KGVA43UKLWXOURLHYBRIEDIDKKK2AGVA, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 139.59.144.22:56836->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716311274 (321h44m54.519775865s) [noise=true , address=77.90.31.12:4590 , piece_id=LV635WNGGSTIAYP6CW5T6756ZS5ZRL5HUPFTH3EVKH3JE7VB4UNQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 139.59.144.22:60484->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716753675 (321h44m54.520218267s) [noise=true , address=77.90.31.12:4592 , piece_id=VIHVVY3HJXCUZWI2U3ZXN2JGLZFXK6MB5NZJKLVRG4WE5YP2RVDQ, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 139.59.144.22:59820->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716937387 (321h44m54.520401984s) [noise=true , address=45.91.116.5:4298 , piece_id=DDM3LFWCP3XJDDILAPCRGRG25XEID2VXHGCN2NJOBPYQCV2GFR5A, seg_pos={0 0}, upload=283130]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 139.59.144.22:52132->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:06.716972021 (321h44m54.520436618s) [seg_pos={0 0}, upload=283130]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Oct 6 23:15:06.823750501 (321h44m54.627215092s) [upload=283130]: Done uploading segments... Oct 6 23:15:06.823805470 (321h44m54.627270074s) [upload=283130]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.606453197s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X...X..X..X............................................] duration: 2.404333275s FAIL (2025-10-06 23:50:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== :14:40.114564120 (321h44m27.918028721s) [upload=283133]: Got next segment. Inline: false Oct 6 23:14:40.114637926 (321h44m27.918102522s) [upload=283133]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 23:14:40.114645588 (321h44m27.918110194s) [upload=283133]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 23:14:40.234782987 (321h44m28.038247566s) [seg_pos={0 0}, upload=283133]: Begin upload segment... Oct 6 23:14:40.234869686 (321h44m28.038334277s) [seg_pos={0 0}, upload=283133]: Done begin upload segment. Oct 6 23:14:40.234988877 (321h44m28.038453472s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235049392 (321h44m28.038513986s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.235610571 (321h44m28.039075214s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.236286136 (321h44m28.039750739s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237036602 (321h44m28.040501193s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.237484234 (321h44m28.040948826s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238009813 (321h44m28.041474414s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.238662655 (321h44m28.042127268s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239160195 (321h44m28.042624779s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239295200 (321h44m28.04275978s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.239764845 (321h44m28.043229424s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240641257 (321h44m28.044105848s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.240760607 (321h44m28.044225207s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241175381 (321h44m28.044639975s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241616623 (321h44m28.045081215s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.241857613 (321h44m28.045322217s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242216650 (321h44m28.045681244s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.242561429 (321h44m28.04602602s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Uploading piece... Oct 6 23:14:40.817451359 (321h44m28.620915951s) [upload=283133]: Next returned nil segment Oct 6 23:14:40.817469197 (321h44m28.620933791s) [upload=283133]: Waiting for error group managing segments... Oct 6 23:14:40.964356063 (321h44m28.767820647s) [noise=true , address=145.79.227.21:4501 , piece_id=GEYOLDIMYZBK4GJ5PNKOQTQLXAXKKOOFOAFFA3UV6HQ77L4TT63A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.967615781 (321h44m28.771080358s) [noise=true , address=145.79.227.32:4834 , piece_id=3JNT3GX5EMGON7RQJKWUXRDDNSHWU35YGHIARPVFFQ4J4J2IHHOA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971520114 (321h44m28.774984697s) [noise=true , address=145.79.227.16:4940 , piece_id=WWWR2LVOH7B2SZAXYMAW7JF4KLAKCJTHAPOVDOD37QJ22SQ4Y3MA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.971634950 (321h44m28.775099534s) [noise=true , address=145.79.227.13:4806 , piece_id=623JR4BIGPX22WONG57VN6U7V7ZLLAG4EIJATDZ4HQDDCZTWZKPA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.972075821 (321h44m28.775540404s) [noise=true , address=145.79.227.41:4094 , piece_id=W62USHBCFXW5XMDKS42J4NUAOZSSM7EVA25MT54UPS4LJUEZRY3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.994068275 (321h44m28.797532859s) [noise=true , address=145.79.227.27:4714 , piece_id=MZA424ADSRXBIBMEPPCWHJPDLRIJIUBV23NZ22HS3K43RWTO35YQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:40.996802013 (321h44m28.800266591s) [noise=true , address=145.79.227.28:4743 , piece_id=6WQFTXPRGMFV3Y5HYAGOKKAFWGNCNCLD43QXVRFC37J3AASMJ7BA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.039669025 (321h44m28.843133624s) [noise=true , address=145.79.227.33:4895 , piece_id=JCKD4RRUZLSB47I4WULRQKH22R42AEMZKUTSKYSSWZXMY7KPMPBQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.549371356 (321h44m29.35283595s) [noise=true , address=64.49.13.26:4956 , piece_id=TY4ZGSVIDE576GMJD374UG2B2HAJ4JJ3QAGH4IDZOXAFO3YOC37A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:14:41.556112439 (321h44m29.359577038s) [noise=true , address=64.49.13.27:4967 , piece_id=6Q4LKMKTS754CDFRVRPE2G7G2WRMONESTQKEFZ7FUTUIPXPVT6ZQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.486669475 (321h45m21.290134069s) [noise=true , address=77.90.31.17:4775 , piece_id=CSMWR3GLQV4ZJZR52K4A6ORLKSGJK47QTOD6H3XOCXJKS45QIW5A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:33.525025992 (321h45m21.32849057s) [noise=true , address=45.91.116.8:4060 , piece_id=45NSYIADUNFCZEK23PEKRJB5AATQAJ5AG24SFVB5THRMZVOPPTFQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:35.522001875 (321h45m23.325466454s) [noise=true , address=77.90.31.10:4522 , piece_id=K4FBSO53SABCWGRYRY4FCGQPSLHNIO6IWEFOCFKPS772UORNWEDA, seg_pos={0 0}, upload=283133]: Done uploading piece. err: <nil> Oct 6 23:15:39.810917110 (321h45m27.61438169s) [noise=true , address=77.90.31.18:4835 , piece_id=ZEIA6RG2GCCFVWEEQWGUFWQGY2UAZ5GPJVBRQOLNN3WJXVUJEZ2A, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:12hAQp2Zr7WSaQSTCZf5pwSyuBHHSrZWq5EVz5hFvSA5ddWebqg, address:77.90.31.18:4835): protocol: rpc: dial tcp 77.90.31.18:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.810972535 (321h45m27.614437116s) [noise=true , address=77.90.31.11:4584 , piece_id=FII3T3RXJLSSBHQI3UC4PUF3EELRPT5ZAUQUG2DRACM76G5AYHGQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1iybcvQgVfqHcCSNFAEVpKZJyciSHbBfeAiuk3uNE2LGq4aLfJ, address:77.90.31.11:4584): protocol: rpc: dial tcp 77.90.31.11:4584: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811092364 (321h45m27.614556942s) [noise=true , address=45.91.116.7:4021 , piece_id=BTKLX4QVZEAXFR2H3WNJVUFNOZY7HEG7PCWNF5L3KECL2HDUXU3Q, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1bXLNG7g5RJc21AjrfHup6LDTd2dDJGWMzrBhVYwUT7fNm8J2Z, address:45.91.116.7:4021): protocol: read tcp 139.59.144.22:51136->45.91.116.7:4021: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811128811 (321h45m27.614593393s) [noise=true , address=77.90.31.12:4617 , piece_id=7QGFJDCAUPQ3OQY3J32A5RPGBXNKQYPSGNSKWEZ2CP6YPZSJNFSQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1PpGinvTbgS3cMHoLaMhhyab7L3urApr3DJRJG4SMzWqw8UAPg, address:77.90.31.12:4617): protocol: read tcp 139.59.144.22:52612->77.90.31.12:4617: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 6 23:15:39.811267995 (321h45m27.614732571s) [noise=true , address=77.90.31.13:4642 , piece_id=K5SJ474XLG4RKZ5OVDELV5EJWCLKE7P7QJDXLQFFF5MY5TE2VEDQ, seg_pos={0 0}, upload=283133]: Done uploading piece. err: ecclient: upload failed (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D, address:77.90.31.13:4642): protocol: write tcp 139.59.144.22:34550->77.90.31.13:4642: use of closed network connection; conn closed: op Oct 6 23:15:39.811289689 (321h45m27.614754266s) [seg_pos={0 0}, upload=283133]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 6 23:15:39.920043283 (321h45m27.723507868s) [upload=283133]: Done uploading segments... Oct 6 23:15:39.920082671 (321h45m27.723547251s) [upload=283133]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.404379631s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)