Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 2.871276999s FAIL (2025-07-26 20:37:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.871304536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.906885153s FAIL (2025-07-26 20:37:10 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.906982106s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.813306811s FAIL (2025-07-26 20:36:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.813319448s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.361177608s FAIL (2025-07-26 20:36:16 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.361221535s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 857.696567ms FAIL (2025-07-26 20:36:10 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 857.730647ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 3.089197363s FAIL (2025-07-26 20:36:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.089226087s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.837283509s FAIL (2025-07-26 20:35:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.83731167s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.921605281s FAIL (2025-07-26 20:35:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.921652523s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 888.567523ms FAIL (2025-07-26 20:35:18 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 888.60207ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.786965602s FAIL (2025-07-26 20:35:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.78698805s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 2.73558445s FAIL (2025-07-26 20:34:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.735614564s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.755718825s FAIL (2025-07-26 20:34:49 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.755763543s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.736649183s FAIL (2025-07-26 20:34:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.736672892s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 840.055953ms FAIL (2025-07-26 20:34:22 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 840.081688ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.358525429s FAIL (2025-07-26 20:34:21 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.358560294s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.99591539s FAIL (2025-07-26 20:34:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.995952619s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 3.513322361s FAIL (2025-07-26 20:33:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.513351457s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.841410562s FAIL (2025-07-26 20:33:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.841477567s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.509144475s FAIL (2025-07-26 20:33:33 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 3.509178379s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.99960936s FAIL (2025-07-26 20:33:16 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.999654252s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 974.965443ms FAIL (2025-07-26 20:33:16 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 975.001089ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.912032715s FAIL (2025-07-26 20:33:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.912062794s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 3.022778441s FAIL (2025-07-26 20:32:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.02281603s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.380313749s FAIL (2025-07-26 20:32:41 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 2.380347139s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 931.004566ms FAIL (2025-07-26 20:32:25 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 931.052597ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 3.863076715s FAIL (2025-07-26 20:32:23 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 3.863108191s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.761607709s FAIL (2025-07-26 20:32:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.761658497s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 3.796293375s FAIL (2025-07-26 20:32:05 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 3.796337062s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 3.308216727s FAIL (2025-07-26 20:32:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.308339197s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 869.67785ms FAIL (2025-07-26 20:31:53 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 869.711737ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 3.230169846s FAIL (2025-07-26 20:31:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.230199014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 3.559194616s FAIL (2025-07-26 20:31:31 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 3.559227247s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 895.53287ms FAIL (2025-07-26 20:31:28 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 895.562883ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.711864353s FAIL (2025-07-26 20:31:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.71188764s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 2.901885023s FAIL (2025-07-26 20:31:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.90192915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 857.422985ms FAIL (2025-07-26 20:30:55 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 857.453759ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 2.452080452s FAIL (2025-07-26 20:30:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.452134592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 3.584784557s FAIL (2025-07-26 20:30:46 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 3.584813674s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.818613614s FAIL (2025-07-26 20:30:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.818640914s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 869.616496ms FAIL (2025-07-26 20:30:20 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 869.644943ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.706567044s FAIL (2025-07-26 20:29:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.706606921s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.381492198s FAIL (2025-07-26 20:29:42 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.381527429s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 7.424143919s FAIL (2025-07-26 20:29:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.424171592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.043480201s FAIL (2025-07-26 20:29:34 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 1.043515401s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 8.161511636s FAIL (2025-07-26 20:29:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.161724629s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 8.537129069s FAIL (2025-07-26 20:29:12 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 8.537174082s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.122358509s FAIL (2025-07-26 20:29:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.122388818s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.111479439s FAIL (2025-07-26 20:28:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.111502481s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001098729s FAIL (2025-07-26 20:28:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001124262s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.000207755s FAIL (2025-07-26 20:28:34 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 1.000239352s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)