Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 3.043189023s FAIL (2025-07-26 21:01:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.043204235s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 2.322195768s FAIL (2025-07-26 21:01:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.322233586s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 2.74683128s FAIL (2025-07-26 21:01:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.746931381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.........................X..................] duration: 2.372722308s FAIL (2025-07-26 21:01:23 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.372753968s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 3.115932983s FAIL (2025-07-26 21:01:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.115968245s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X.........................X.................] duration: 3.566349092s FAIL (2025-07-26 21:00:43 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 3.566383187s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.698130545s FAIL (2025-07-26 21:00:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.698158424s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 2.840223578s FAIL (2025-07-26 21:00:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.840258142s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 3.104879072s FAIL (2025-07-26 21:00:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.105014191s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 3.196289229s FAIL (2025-07-26 21:00:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.196322378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X.........................X................] duration: 3.224280931s FAIL (2025-07-26 20:59:44 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 3.224346885s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 2.389029586s FAIL (2025-07-26 20:59:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.389058737s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X.........................X...............] duration: 2.71415423s FAIL (2025-07-26 20:59:20 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.714192913s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.889167964s FAIL (2025-07-26 20:59:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.889195699s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X.........................X..............] duration: 3.392672784s FAIL (2025-07-26 20:58:50 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 3.392704072s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.657966464s FAIL (2025-07-26 20:58:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.657989951s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 2.450389593s FAIL (2025-07-26 20:58:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.45043353s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.789037459s FAIL (2025-07-26 20:58:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.789073223s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 2.455712115s FAIL (2025-07-26 20:58:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.455745345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X.........................X.............] duration: 2.941514187s FAIL (2025-07-26 20:58:04 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.941625131s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.789202261s FAIL (2025-07-26 20:57:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.789242038s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 2.426775714s FAIL (2025-07-26 20:57:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.426810371s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.700919413s FAIL (2025-07-26 20:57:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.700946767s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.........................X............] duration: 2.447352192s FAIL (2025-07-26 20:56:56 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.447384044s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 2.60797558s FAIL (2025-07-26 20:56:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.608009327s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X.........................X...........] duration: 3.838738363s FAIL (2025-07-26 20:56:25 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 3.838775989s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.55544497s FAIL (2025-07-26 20:55:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.555486171s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.........................X..........] duration: 2.468378032s FAIL (2025-07-26 20:55:39 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.468481963s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 2.303390554s FAIL (2025-07-26 20:55:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.3034233s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.720237492s FAIL (2025-07-26 20:55:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.720278663s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.659669555s FAIL (2025-07-26 20:54:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.659694783s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 2.747476997s FAIL (2025-07-26 20:54:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.747502662s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X.........................X.........] duration: 2.626570231s FAIL (2025-07-26 20:54:30 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.626597591s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.680160708s FAIL (2025-07-26 20:54:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.680200019s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.647694259s FAIL (2025-07-26 20:53:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.64781317s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 2.370902046s FAIL (2025-07-26 20:53:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.370952529s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.........................X........] duration: 2.781122062s FAIL (2025-07-26 20:53:23 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.781151665s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 2.734711173s FAIL (2025-07-26 20:53:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.734742853s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.........................X.......] duration: 3.392245283s FAIL (2025-07-26 20:52:49 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 3.392281386s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.701055887s FAIL (2025-07-26 20:52:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.701140597s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 2.459146452s FAIL (2025-07-26 20:52:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.459179505s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.774420504s FAIL (2025-07-26 20:52:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.77444895s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.........................X......] duration: 2.308719684s FAIL (2025-07-26 20:51:53 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.30874901s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.946446303s FAIL (2025-07-26 20:51:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.946472514s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 3.898884467s FAIL (2025-07-26 20:51:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.898917802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 2.422036774s FAIL (2025-07-26 20:51:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.422133453s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.782091967s FAIL (2025-07-26 20:51:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.782109849s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.672977483s FAIL (2025-07-26 20:50:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.673012086s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X.........................X.....] duration: 2.637918804s FAIL (2025-07-26 20:50:41 UTC) orbiter warning: uplink: object not found ("orbiter_us_0/4M") Duration: 2.637963382s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 2.389072199s FAIL (2025-07-26 20:50:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== rue , address=45.91.116.6:4975 , piece_id=VG5RNI2Z6A6FGM7I6WHCNJJIMZUNXKEXYNT367PJEN756GSYLXCQ, seg_pos={0 0}, upload=54545]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:05.089846946 (56h52m36.280145528s) [seg_pos={0 0}, upload=54545]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 26 20:28:05.204635709 (56h52m36.394934281s) [upload=54545]: Done uploading segments... Jul 26 20:28:05.204661999 (56h52m36.394960579s) [upload=54545]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 26 20:28:26.499846533 (56h52m57.690145103s) [upload=54558]: Starting upload Jul 26 20:28:26.499918833 (56h52m57.690217403s) [upload=54558]: Done starting upload Jul 26 20:28:26.499982310 (56h52m57.690280894s) [upload=54558]: Uploading segments... Jul 26 20:28:26.500267631 (56h52m57.690566217s) [upload=54558]: Got next segment. Inline: false Jul 26 20:28:26.500323092 (56h52m57.690621681s) [upload=54558]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 20:28:26.500349824 (56h52m57.690648517s) [upload=54558]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 20:28:26.611161117 (56h52m57.801459686s) [seg_pos={0 0}, upload=54558]: Begin upload segment... Jul 26 20:28:26.611211593 (56h52m57.801510173s) [seg_pos={0 0}, upload=54558]: Done begin upload segment. Jul 26 20:28:26.611242346 (56h52m57.801540931s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.611787891 (56h52m57.802086479s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612150870 (56h52m57.802449454s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612502942 (56h52m57.802801528s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.612827744 (56h52m57.80312633s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613395709 (56h52m57.803694297s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.613836447 (56h52m57.804135041s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614376380 (56h52m57.804674968s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614682063 (56h52m57.804980645s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.614954123 (56h52m57.805252705s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615254738 (56h52m57.805553321s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615524618 (56h52m57.805823199s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.615836489 (56h52m57.806135079s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616131383 (56h52m57.806429964s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616416900 (56h52m57.806715483s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:26.616716123 (56h52m57.807014706s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Uploading piece... Jul 26 20:28:27.286742660 (56h52m58.477041242s) [upload=54558]: Next returned nil segment Jul 26 20:28:27.286763668 (56h52m58.477062307s) [upload=54558]: Waiting for error group managing segments... Jul 26 20:28:27.422423022 (56h52m58.612721587s) [noise=true , address=104.36.82.4:4801 , piece_id=YPVDYP6TGDVT4E34GWGTPECGZJGIQ3P5A6G57TFJZGFQRLSCLQOA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.550290566 (56h52m58.740589135s) [noise=true , address=104.36.82.13:4187 , piece_id=XLSRJLP5W5AWXAPJNSEND2KYHN7K2LUPFAL5FOSZUF7KB4OHG2CA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.758451628 (56h52m58.948750195s) [noise=true , address=77.90.31.17:4772 , piece_id=32G6AMIJKMU5VDXP3ZSSZ7SY33KDA23K6X3VBZQGEBNSC67CA7WA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.763539754 (56h52m58.953838348s) [noise=true , address=77.90.31.11:4553 , piece_id=RMNEKQ2PCBE2XBZOJDNBT6DBAHRSNNRXP5G7ADW72UJKABXND4IQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.764479628 (56h52m58.954778197s) [noise=true , address=77.90.31.17:4770 , piece_id=W2NFAQBOT4BZSVU3UJDY4AJDSHA23SZWXEAPNUJ7NKFAMPP6H2KQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770394756 (56h52m58.960693336s) [noise=true , address=77.90.31.14:4661 , piece_id=ONNRZLCBQ4CO63XLWSL3OTTDFJ4RLZRG4LFOGYXHJCQRGGIE6L7A, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.770582294 (56h52m58.960880874s) [noise=true , address=77.90.31.12:4590 , piece_id=KBLBOWE7S6FJWKRUVSSM4NXQNMBYJV3F6CAU5CF4KU5A2GQIFCAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.774778478 (56h52m58.965077045s) [noise=true , address=77.90.31.18:4807 , piece_id=KXNRMEBIRPDBXTEB3MQLLWS74DGNKZLOFIBT3UJ6DXSDU4GUK5LA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.784509808 (56h52m58.974808374s) [noise=true , address=77.90.31.10:4517 , piece_id=KK57275YUDHYLD4ORJFC5XX2CWQXHEFB3LHHWHHSJYUR2U3SBRLA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:27.794987896 (56h52m58.985286468s) [noise=true , address=77.90.31.16:4734 , piece_id=3FEPD2DWABI3TUGTVRNNKDLQ43TBMEWGWZYXR6RNDEJNEUNEFROQ, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:28.312192063 (56h52m59.502490639s) [noise=true , address=104.36.82.10:4079 , piece_id=SJAJ57TGXXGIA7QKYDP3O42T7LSI76E3HF6TDNCMBEYTRU6AVWNA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: <nil> Jul 26 20:28:56.210272411 (56h53m27.40057099s) [noise=true , address=45.91.116.14:4191 , piece_id=YMFQRI7HQ7JIONHUUBXS34NAQDDHSMDRQBMVWNJFO5G7Y6EDILBA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: write tcp 139.59.144.22:50304->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210363923 (56h53m27.400662505s) [noise=true , address=45.91.116.12:4226 , piece_id=DLPQ24VJRGDKNEZW5V6WXHDJ2L6QPFWXOCE5772ALHPCGT3VSQAA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210426568 (56h53m27.400725148s) [noise=true , address=45.91.116.14:4190 , piece_id=ERV2BJGSJJ4K6OEIWQ4DF7VQDOOKJ4HPEDYSWDNXHKLD3U6WFWRA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210483041 (56h53m27.40078162s) [noise=true , address=45.91.116.11:4118 , piece_id=D44O4XCV5G3L3WIXVKAAF2QAIMK6DYMKCGMJ5QO2KTLKUSAXJMMA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 139.59.144.22:35884->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 20:28:56.210617764 (56h53m27.400916346s) [noise=true , address=45.91.116.13:4153 , piece_id=SDVID3KRFGBD3QFYFW5CJSEY7ACUCYV4E7AD5TVE7JWUG6434HHA, seg_pos={0 0}, upload=54558]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: context deadline exceeded; context deadline exceeded Jul 26 20:28:56.210643671 (56h53m27.400942269s) [seg_pos={0 0}, upload=54558]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 20:28:56.316670597 (56h53m27.506969171s) [upload=54558]: Done uploading segments... Jul 26 20:28:56.316701523 (56h53m27.507000104s) [upload=54558]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.38910442s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)